Aluminum alloy microstructure image segmentation method based on uncertainty estimation technology

Abstract

The invention discloses an aluminum alloy microstructure image segmentation method based on an uncertainty estimation technology. The method comprises the following steps: obtaining a preheated aluminum alloy metallographic image sample; dividing the preheated aluminum alloy metallographic image sample data set to obtain a training set and a test set; enhancing the aluminum alloy metallographic image sample data in the training set to obtain an enhanced aluminum alloy metallographic image sample training set; constructing a segmentation network model based on Beta distribution, wherein the segmentation network model is used for carrying out pixel-level segmentation on silicon particles in the image; training a segmentation network model based on Beta distribution by using the enhanced aluminum alloy metallographic image sample training set data to obtain a trained segmentation network model based on Beta distribution; and inputting test set data into the trained Beta distribution-based segmentation network to predict the probability of silicon particles contained in the image, and assisting experts to obtain qualitative conclusions and quantitative indexes of the microstructure.

Description

Technical Field

[0001] The invention belongs to the field of fully automated products and relates to an aluminum alloy microstructure image segmentation method based on uncertainty estimation technology. Background Art

[0002] In the material manufacturing process, the analysis of microstructure images plays a vital role in understanding the internal structure of the material, evaluating its quality and optimizing the production process. These images provide important information about the internal microstructure of the material and are key to ensuring the quality and performance of alloy products. However, traditional manual analysis methods are not only time-consuming and labor-intensive, but also easily affected by human factors. Therefore, reducing the need for manual dependence and using computer vision technology for automated analysis has become an important direction in research and industrial applications.

[0003] At present, the research on microscopic tissue image analysis mainly focuses on several core tasks: image classification [1-2] , Image Segmentation [3-4] and boundary detection [5] . Image classification tasks usually involve quality assessment and microstructure type recognition, for example, determining whether an image contains a specific type of tissue structure. Image segmentation tasks include semantic segmentation (dividing an image into regions with specific meanings) and instance segmentation (identifying and separating different instances in an image). Boundary detection tasks focus on identifying details such as grain boundaries in microstructures. To address these issues, researchers have adopted a variety of methods, including traditional image processing techniques (such as Otsu thresholding) and [6] ), machine learning methods (such as support vector machines, random forests, K nearest neighbor methods [7] )wait.

[0004] With the rise of deep learning technology, scholars have gradually turned their attention to using deep learning methods to improve microscopic tissue image analysis. Deep learning models, such as UNet and Mask RCNN, have shown excellent performance in image segmentation and instance detection. For example, Breumier [8] et al. introduced unknown items as additional annotations to train UNet [9] The Mask RCNN is used to segment the three microstructures of bainite, ferrite and martensite in the image.

[10] and Cascade Mask RCNN and other advanced deep learning frameworks [11-12] It has been widely used, fully demonstrating the capabilities of deep learning in microstructure detection problems.

[0005] Although deep learning technology has shown significant advantages in visual tasks, existing methods still have some shortcomings. Current research and applications often fail to fully consider the uncertainty issues in microscopic tissue images, such as similar appearance and blurred boundaries. Many existing deep learning models and strategies may avoid or simplify these difficulties when facing these complex scenarios, thus affecting the final analysis results. In addition, existing technical solutions lack targeted adjustments to the characteristics of microscopic tissue images, especially in the introduction of Bayesian methods and uncertainty estimation techniques, which still need to be improved. [13-14] .

[0006] Given the importance of microstructure images in the alloy manufacturing process, the industry is very cautious about the practical application of deep learning models. How to effectively capture the uncertainty in microstructure images and improve the robustness and accuracy of the model has become an important challenge in current research. Future research should focus on developing more accurate deep learning technologies that can not only handle complex practical scenarios, but also provide more reliable analysis results, thereby promoting the widespread application of intelligent detection technology and improving production efficiency and product quality.

[0007] In general, in-depth exploration of the application of deep learning in microstructure image analysis, especially in dealing with uncertainty issues, will help achieve the accuracy and reliability of automated detection and have a profound impact on the development of materials science and manufacturing.

[0008] The size, shape, and distribution of microstructures dominate the macroscopic properties of materials, and analyzing their images is an important means of detection in production. These images can reveal important microstructure information at different production stages, are the key basis for testing alloy quality, and are also of great significance for revealing the mechanical properties of materials. However, since the boundaries of microstructures are blurred after imaging, and the appearance of some particles of different phases is very similar, there is inherent uncertainty in the images, and the complexity of intelligent analysis can be imagined. Traditional deep learning methods often give overconfident predictions, so the current industry still relies mainly on manual analysis by experts. Summary of the invention

[0009] In order to solve the above problems, the technical solution adopted by the present invention is: a method for segmenting aluminum alloy microstructure images based on uncertainty estimation technology, comprising the following steps: a method for segmenting aluminum alloy microstructure images based on uncertainty estimation technology, comprising the following steps:

[0010] Acquire metallographic image samples of aluminum alloy after preheat treatment to form a metallographic image sample data set of aluminum alloy after preheat treatment;

[0011] The sample data set of aluminum alloy metallographic images after preheat treatment is divided into a training set and a test set;

[0012] The aluminum alloy metallographic image training sample data in the training set is enhanced to obtain an enhanced aluminum alloy metallographic image sample training set;

[0013] Construct a segmentation network model based on Beta distribution to perform pixel-level segmentation of silicon particles in images;

[0014] The enhanced aluminum alloy metallographic image sample training set data is used to train the segmentation network model based on Beta distribution, and a trained segmentation network model based on Beta distribution is obtained;

[0015] The test set data is input into the trained segmentation network based on Beta distribution to predict the probability of silicon particles contained in the image.

[0016] Furthermore, the process of constructing a segmentation network model based on Beta distribution for pixel-level segmentation of silicon particles in an image is as follows:

[0017] The following formula is used to predict the uncertainty of aluminum alloy metallographic image samples:

[0018]

[0019] Here, μ represents the parameter of the distribution, p(μ|x * ,ω) represents the uncertainty of the distribution; p(ω|X,Y) represents the uncertainty of the model parameters; p(y * |x * , ω) represents the metallographic image x of the tested aluminum alloy * The prediction results;

[0020] Beta distribution is used to implement the second term of the integral in formula (8), and its probability distribution is used to describe the uncertainty of Bernoulli distribution. The probability density distribution is as follows:

[0021]

[0022] Among them, α and β are the two coefficients of Beta distribution, and the denominator Γ(·) represents the Gamma function;

[0023] The prior distribution of parameter θ is regarded as Dirac distribution ω~δ(θ), that is, it is meaningful only when the variable ωω=θ. At this time, the posterior distribution of the parameter is directly equivalent to the prior distribution and has nothing to do with the data. The integral of dω in equation (8) will disappear, and equation (8) can be simplified to obtain the following equation:

[0024] p(y * |x * , X, Y) = ∫p(y* |μ)p(μ|x * ,θ)dμ (10)

[0025] When there is a metallographic sample x of aluminum alloy to be tested * When the segmentation network model based on Beta distribution is input, formula (10) is equivalent to the integral of the Bernoulli distribution probability μ, where μ obeys the Beta distribution. Once the two shape parameters α and β of the Beta distribution are determined, the closed-form solution of equation (10) can be directly obtained through expectation. While evaluating the uncertainty, the prediction ability of the microstructure segmentation task is improved. The positive category is silicon particles, and the negative category is aluminum matrix and interference. The prediction results are as follows:

[0026]

[0027]

[0028] Furthermore, the parameter θ of the segmentation network based on the Beta distribution uses a unique loss function during the training process, and the maximum likelihood estimation equation of the parameter θ is L MLE (x, y; θ) is expressed as:

[0029]

[0030] Its closed form is:

[0031]

[0032] Here, Bern(·) represents Bernoulli distribution, and ψ(·) is the derivative of the logarithm of the Γ(·) function.

[0033] Furthermore: the additional constraint of the segmentation network model based on Beta distribution adopts a penalty term L based on the truncation coefficient NormC (x, C; θ), as shown below:

[0034]

[0035] Among them, parameter C is a preset hyperparameter.

[0036] Further: the loss function of the segmentation network model based on Beta distribution is expressed as follows:

[0037] L Beta =L MLE (x,y;θ)+τL NormC (x,C;θ) (16)

[0038] Among them, the hyperparameter τ is used to balance the two loss functions, where the likelihood function L MLE(·) is used to fit the segmentation network model based on Beta distribution with supervised information, and obtain a closed-form solution through mathematical derivation, which is then integrated into the back propagation process of the neural network. NormC (x, C; θ) ensures the numerical stability of the optimization process through the truncation coefficient. The two work together to ensure that the model outputs a reasonable probability and effectively estimates the uncertainty.

[0039] Further: the process of using the enhanced aluminum alloy metallographic image sample training set data to train the segmentation network model based on Beta distribution to obtain the trained segmentation network model based on Beta distribution is as follows:

[0040] Obtain aluminum alloy microstructure image {X, Y}, truncation coefficient C, and balance coefficient τ;

[0041] Randomly initialize the parameter θ;

[0042] Collect sample-label pairs (x, y) from the aluminum alloy microstructure image set (X, Y);

[0043] In the output stage of the Beta distribution segmentation network model, the output result after the terminal convolutional layer is transformed, that is, exp(|·|)=α,β;

[0044] Using the closed form of the maximum likelihood estimation equation, calculate the maximum likelihood-based loss function L MLE (·);

[0045] When the output α+β>C after the transformation of the segmentation network model based on the Beta distribution: use the penalty term L NormC (x, C; θ), introduce the penalty loss L NormC (·);

[0046] Combine the loss function of the segmentation network model based on Beta distribution with the balance coefficient τ to calculate the overall loss L of the segmentation network model based on Beta distribution Beta ;

[0047] The model parameter θ is updated according to the gradient of the segmentation network model based on Beta distribution to realize the training of the segmentation network model based on Beta distribution.

[0048] Furthermore: the segmentation network model based on Beta distribution adopts an improved UNet deep convolutional network;

[0049] The improved UNet deep convolutional network contains feature maps of three scales, namely, the original image size is 376×376, the size after one downsampling is 188×188, and the size after two downsampling is 94×94;

[0050] In the encoder part, as the scale decreases, the number of features after the convolution operation gradually increases, which are 64, 128, and 128 respectively; through this gradual downsampling method, more abstract and deep features of the input data are gradually extracted;

[0051] In the decoder part, the downsampled features are gradually upsampled by bilinear interpolation until they are restored to the original image size. After each upsampling, the jump connection design of UNet integrates the feature map of the corresponding encoder with the feature map of the decoder and performs a convolution operation.

[0052] Finally, the result with the same size as the original image is output and converted into a probability value through the expectation of Beta distribution, thereby achieving the task of segmenting silicon particles in the preheating metallographic image.

[0053] An aluminum alloy microstructure image segmentation device based on uncertainty estimation technology, comprising:

[0054] Acquisition module: used to acquire metallographic image samples of aluminum alloy after preheating treatment, forming a data set of metallographic image samples of aluminum alloy after preheating treatment;

[0055] Division module: used to divide the metallographic image sample data set after preheating treatment into a training set and a test set;

[0056] Enhancement module: used to enhance the aluminum alloy metallographic image sample data in the training set to obtain an enhanced aluminum alloy metallographic image training set;

[0057] Building module: used to build a segmentation network model based on Beta distribution for pixel-level segmentation of silicon particles in images;

[0058] Training module: Use the enhanced aluminum alloy metallographic image training set data to train the segmentation network model based on Beta distribution to obtain a trained segmentation network model based on Beta distribution;

[0059] Prediction module: Input the test set data into the trained segmentation network based on Beta distribution to predict the probability of silicon particles contained in the image.

[0060] A computer device comprises: a processor and a memory, wherein the memory stores a program module, and the program module runs on the processor to implement any one of the methods described above.

[0061] An aluminum alloy microstructure image segmentation device based on uncertainty estimation technology, comprising:

[0062] Memory;

[0063] processor; and computer program;

[0064] The computer program is stored in the memory and is configured to be executed by the processor to implement any one of the methods described.

[0065] The aluminum alloy microstructure image segmentation method based on uncertainty estimation technology provided by the present invention has the following advantages:

[0066] (1) The introduction of uncertainty estimation technology improves the prediction effect of the method on the microstructure segmentation task: In order to improve the prediction accuracy of the preheat treatment metallographic image, a segmentation network model based on Beta distribution is adopted. It embeds uncertainty estimation ability directly in the model by fitting the shape parameters of Beta distribution. Experimental results show that compared with the traditional UNet method, this method has significant advantages in accuracy, intersection-over-union ratio and Dice score. In particular, the segmentation network model based on Beta distribution proposed by us has the best performance among all tested methods with an accuracy of 0.9749 and a Dice score of 0.9075. This not only shows the effectiveness of the segmentation network model based on Beta distribution in image segmentation tasks, but also proves the importance of uncertainty estimation in complex industrial application scenarios.

[0067] (2) Uncertainty estimation methods enhance the reliability and robustness of model predictions: In the application of deep learning models, models that rely solely on high accuracy are often insufficient to cope with the complexity of actual production environments. In order to better understand the predictive ability of the model, the effects of different uncertainty estimation methods are evaluated by analyzing the relationship between misclassified pixels and confidence. The results show that the model that introduces uncertainty estimation can more accurately identify the uncertain areas in the prediction results, thereby improving the overall prediction reliability. In the experiment, the Beta distribution segmentation network model performed outstandingly in both AUROC and AUPR scores, especially in AUROC, where it achieved a high score of 0.9721, showing excellent ability to correct output probabilities. The MaxP and NormEN strategies did not differ much in these numerical indicators, but after visualization, the uncertainty heat map generated by the NormEN strategy was more obvious, indicating higher uncertainty areas. It is worth noting that in the silicon particle segmentation task, the areas with high uncertainty are usually concentrated near the boundaries of the silicon particles and interference items, which is consistent with the analysis results of experts in the field, further verifying the reliability of the method.

[0068] (3) The segmentation network model with Beta distribution has higher practicality in real-world applications: In industrial applications, the spatial complexity and time complexity of the model directly affect its feasibility in the production environment. Although deep integration and dataset splitting methods based on integration ideas can significantly improve model performance, the cost is a significant increase in the storage requirements of the model. As the number of base classifiers increases, the number of model parameters increases exponentially, making its deployment in practical applications more difficult. Similarly, the variational Bayesian method is computationally time-consuming due to its reliance on the sampling process, which is a considerable challenge in application scenarios that require fast response. In contrast, the segmentation network model with Beta distribution proposed by us is designed with the needs of practical applications in mind. By adding only a small number of parameters at the end of the model to fit the shape parameters of the Beta distribution, the method successfully achieves effective uncertainty estimation without significantly increasing the complexity of the model. At the same time, due to the nature of the Beta distribution, the expected value of the prediction can be directly calculated without the need for tedious sampling operations like the Bayesian method, which greatly improves the testing efficiency of the model. The prediction speed of the segmentation network model based on Beta distribution reaches 17.02FPS, which is very close to the 17.22FPS of the original UNet network. This means that in actual industrial environments, the segmentation network model based on Beta distribution can not only provide high-quality prediction results, but also be quickly deployed at a lower computing cost, showing extremely high practical value.

[0069] Aiming at the specific problems of similar morphology of particles of different phases in microstructure images and blurred boundaries of microstructures, the present invention studies the deep learning method under the condition of coexistence of multiple similar particles, proposes to introduce uncertainty estimation technology to effectively detect and analyze the micron-level microstructures involved, applies uncertainty estimation technology to microstructure analysis, and proposes a segmentation network model based on Beta distribution to achieve segmentation tasks. It effectively captures the uncertainty of silicon particle prediction, realizes the deep learning method for microstructure analysis in complex scenarios, solves the bottleneck problems of low recognition accuracy and poor robustness of traditional image processing methods, and improves the reliability and robustness of the method. Aiming at the actual needs of intelligent detection of enterprises, the present invention takes aluminum alloy microstructure images as the research object, designs intelligent analysis methods for the bottleneck problem of uncertainty, and has clear practical value for improving production efficiency and reducing labor costs.

[0070] According to the actual needs of the production process, the present invention explores effective visual representations in microstructure analysis scenarios, explores deep learning methods that capture uncertainty, and assists experts in obtaining qualitative conclusions and quantitative indicators of microstructures. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0072] Figure 1 It is a framework diagram of the segmentation method of this application;

[0073] Figure 2 It is the microstructure image of aluminum alloy after preheat treatment, where (a) is the microstructure image after preheat treatment, and (b) is the true value image of silicon particle segmentation;

[0074] Figure 3 The following are three possible prediction results, where (a) is a deterministic prediction, (b) is data uncertainty, and (c) is epistemic uncertainty.

[0075] Figure 4 is a variational Bayesian network diagram, where (a) is a regular network and (b) is a Bayesian network;

[0076] Figure 5 It is a structural diagram of the segmentation network model based on Beta distribution adopted in this application;

[0077] Figure 6 The prediction results of silicon particle segmentation in aluminum alloy microstructure images, where (a) is the metallographic image, (b) is the prediction result of the original Unet network, and (c) is the prediction result of the segmentation network model based on Beta distribution;

[0078] Figure 7 It is a visualization of the uncertainty estimation of the segmentation network model based on Beta distribution in microscopic tissue images, where (a) is the misclassified pixel map, (b) is a visualization of the uncertainty estimation of the segmentation network model based on Beta distribution in the MaxP calculation method, and (c) is a visualization of the uncertainty estimation of the segmentation network model based on Beta distribution in the NormEn calculation method.

[0079] Figure 8 is a plot of model capacity versus computational efficiency for microstructure segmentation methods. DETAILED DESCRIPTION

[0080] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0081] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0082] Figure 1 It is a framework diagram of the segmentation method of this application;

[0083] A method for segmenting aluminum alloy microstructure images based on uncertainty estimation technology comprises the following steps:

[0084] S1: Acquire metallographic image samples of aluminum alloy after preheat treatment to form a metallographic image sample data set of aluminum alloy after preheat treatment;

[0085] S2: dividing the aluminum alloy metallographic image sample data set after preheat treatment into a training set and a test set;

[0086] S3: enhancing the aluminum alloy metallographic image sample data in the training set to obtain an enhanced aluminum alloy metallographic image sample training set;

[0087] S4: constructing a segmentation network model based on Beta distribution to perform pixel-level segmentation of silicon particles in the image;

[0088] S5: using the enhanced aluminum alloy metallographic image sample training set data to train the segmentation network model based on Beta distribution, to obtain a trained segmentation network model based on Beta distribution;

[0089] S6: Input the test set data into the trained segmentation network based on Beta distribution to predict the probability of silicon particles contained in the image.

[0090] Steps S1 / S2 / S3 / S4 / S5 / S6 are executed in sequence;

[0091] This method uses uncertainty estimation to study the metallographic image of aluminum alloy after preheat treatment. The research object is the metallographic image of aluminum alloy after preheat treatment. Preheat treatment is the process of heating and heat preservation of aluminum-silicon-magnesium alloy ingots. The main purpose of this stage is to convert the fiber-net eutectic silicon in the cast state into fine silicon particles, so as to prepare for the subsequent heat treatment processes such as solid solution and aging. Therefore, it is very important to observe the morphology of silicon particles in aluminum alloy after preheat treatment to ensure the processing effect.

[0092] In practical applications, technicians usually use metallographic microscopes to study the statistical morphology of microscopic particles inside aluminum alloys, such as Figure 2 As shown, (a) is the preheat treatment microstructure image, and (b) is the silicon particle segmentation truth table; at this time, in the metallographic image, the observed microstructure can be divided into four categories: aluminum dendrites that occupy the main part, round silicon particles, and irregular gray Al8FeMg3Si6 and black Mg2Si. Generally speaking, technicians need to select the silicon particles and mark them one by one, and then obtain statistical characteristics to examine the effect of the predicted treatment process. Obviously, this process takes a lot of time and effort, and due to the existence of two interference items with similar grayscales, this type of annotation requires experienced experts to complete, which greatly increases the manufacturing and testing costs of the enterprise. Therefore, it is a challenging task to develop an automated silicon particle segmentation method for aluminum alloy metallographic images containing interference items.

[0093] The present invention aims at the problem of aluminum alloy microstructure image segmentation, fully considers the influence of interference items with similar appearance and blurred boundaries in the image, designs a classic deep learning model, and introduces a variety of uncertainty estimation methods to achieve high-performance and robust prediction effects. Specifically, this problem is first abstracted as a semantic segmentation task. Considering the importance of silicon particle size for testing the effect of preheat treatment, this task is simplified to a pixel-level binary classification task, that is, silicon particles are regarded as foregrounds, while aluminum matrix and two interference items are regarded as backgrounds. Next, considering the characteristics of the silicon particle segmentation task itself, a deep convolutional network is selected to implement this visual task. However, as mentioned earlier, due to the presence of a variety of interference items with similar appearances in microstructure images, conventional convolutional networks are not completely suitable for this scenario, especially considering the importance of microstructure detection in production and manufacturing, it is necessary to introduce relevant training strategies. To this end, the present invention designs a variety of uncertainty estimation methods based on convolutional networks, reasonably corrects the output probability, improves the robustness of model prediction, and reduces the impact of uncertainty.

[0094] Regarding enhancing the aluminum alloy metallographic image sample data after preheating treatment, obtaining an enhanced aluminum alloy metallographic image sample data set;

[0095] For image acquisition and data set enhancement, high-quality image collection and augmentation using methods such as rotation and flipping provide rich training data for subsequent deep learning models, which helps to improve the model's training effect and prediction accuracy.

[0096] The data of the present invention comes from the aluminum-silicon-magnesium alloy after preheat treatment. After a series of operations such as sampling, grinding, polishing and etching, it is observed using a metallographic microscope, and a total of 198 metallographic images with a size of 376×376 are obtained. Each image is annotated at the pixel level, with the positive category being silicon particles and the negative category being the aluminum matrix and interference items. In the process of the evaluation method, these images are randomly divided into 5 parts, 4 of which are used each time to train the deep model, and the remaining part is used to verify the effect of the model, thereby cross-validating different methods. During training, according to the general logic of deep learning, the training image samples are augmented by flipping, rotating and other methods, and the image data sent to the deep model is finally expanded to 8 times the original, and the effectiveness of the model is evaluated in combination with the test set.

[0097] Secondly, considering the success of convolutional networks in solving computer vision tasks, the present invention designs a deep convolutional network to solve the microstructure image segmentation task. This is a segmentation network model based on Beta distribution based on an encoder-decoder structure. This structure can effectively handle image segmentation tasks and is suitable for fine pixel-level segmentation of silicon particles. Next, in order to deal with the problem of grayscale proximity of several interference items, three different uncertainty estimation methods are introduced. They are respectively the integration method

[15] , Bayesian Deep Network [16-17] and Beta-distributed segmentation network models. Among them, the integration method uses multiple models to make predictions at the same time to achieve uncertainty estimation; the Bayesian deep network models the probability of model parameters and focuses on capturing the uncertainty of the model; the Beta-distributed segmentation network model estimates from the perspective of distribution uncertainty. Among them, in this invention, the first two techniques have been proven to be effective in common problems. Finally, we will conduct a series of experiments to evaluate the actual effects of these uncertainty estimation methods. Through these experiments, we can verify the performance of the three methods in actual scenarios.

[0098] The process of the micro-organization uncertainty estimation method based on ensemble learning is as follows:

[0099] Deep neural networks have performed well in many visual tasks, but they often face the challenge of overfitting in supervised learning. This phenomenon causes the model to be overly dependent on training data, resulting in overconfident judgments in practical applications and difficulty in accurately assessing uncertainty. This situation affects the effectiveness of deep learning in key areas, such as microstructure image analysis in materials science. To solve this problem, ensemble learning technology provides an effective method. Specifically, by training multiple deep neural networks and combining the prediction results of each network for the sample, the negative impact of overfitting can be reduced and the generalization ability of the model can be improved. This method not only provides more reliable prediction results, but also enhances the robustness of the model when facing complex data. When we want to make a prediction for a sample, we make multiple copies of it and feed them into different networks at the same time, and then examine the results of the overall prediction, such as Figure 3 As shown. Among them, (a) is the deterministic prediction, (b) is the data uncertainty, and (c) is the epistemic uncertainty;

[0100] exist Figure 3 In , we present the possible prediction results of ensemble learning in the same image, and the color depth of the image represents the density of its prediction. From left to right, three different prediction results are listed in turn. The first case is the ideal state, that is, all prediction results are concentrated in the same category, such as the silicon particles represented at the top here. At this time, the prediction results of multiple independently trained models are very close. In image segmentation tasks, this situation usually occurs in areas with clear contours or categories, such as pixels inside silicon particles or clean background pixels. When encountering more complex areas, such as pixels near interference items or boundaries, the model's prediction results are often concentrated in the central area, as shown in the second sub-figure. This situation is called data uncertainty. The last case is more complicated and involves epistemic uncertainty or model uncertainty. It usually occurs when the model is overfitted or the test sample and the training sample do not meet the independent and identically distributed conditions. At this time, the prediction results of multiple models in ensemble learning will be more scattered, indicating that there is a high degree of epistemic uncertainty. From the perspective of mathematical expectation, when a sample x is * When making forecasts, these two types of uncertainty can be related using the following formula.

[0101]

[0102] Among them, the symbols X, Y represent supervised sample pairs, and ω is a model-related variable.

[0103] In the designed ensemble learning, the variable ω in formula (1) refers to different convolutional networks. * |x * ,ω) represents a single model for the test image x *The prediction result is further that p(ω|X, Y) is essentially a pre-given equal probability category distribution under ensemble learning. Under this condition, formula (1) can be further rewritten as:

[0104]

[0105] Among them, m i represents the deep model with serial number i, and h represents the number of models in ensemble learning.

[0106] Next, we will introduce the implementation process of the microscopic tissue image segmentation task using the ensemble method. In order to obtain a diverse deep model, we mainly consider two implementation methods, namely the deep ensemble method.

[15] And the dataset splitting method. The former is mainly achieved by initializing random seed points of different models, while the latter divides the dataset into several independent parts to achieve the diversity of deep models. The algorithm is shown in Table 1.

[0107] Table 1 Ensemble learning algorithm for aluminum alloy microstructure images

[0108]

[0109]

[0110] In Table 1, two integration methods are implemented at the same time, namely deep integration

[15] The deep integration method and the integration method based on data set splitting are distinguished by the identifier Sign. As you can see, the deep integration method is different from the integration algorithm in machine learning.

[18] There is a significant difference, the latter mostly rely on different training subsets to achieve model diversity

[19] , while the models in deep ensemble use the same supervised data, and their diversity comes from the initialization stage of the model. In solving the task of aluminum alloy microstructure segmentation, the two methods will be implemented and compared simultaneously.

[0111] The process of the microstructure uncertainty estimation method based on variational Bayesian network is as follows:

[0112] A significant drawback of using ensemble learning to achieve uncertainty estimation is that it increases the consumption of storage space and the time of prediction. For the problem of microscopic tissue image analysis, introducing uncertainty in model parameters is another effective solution. This method was proposed earlier than the ensemble method and has a more solid theoretical basis.

[20] , whose advantage is that the overfitting problem can be solved by a single model. To achieve this effect, in the Bayesian network, the model parameters are no longer fixed values ​​in the traditional sense, but obey a specific distribution, such as Figure 4As shown, (a) is a conventional network and (b) is a Bayesian network;

[0113] Figure 4 The schematic diagrams of conventional neural networks and Bayesian neural networks are shown respectively. It can be found that the model parameters of the latter obey a certain distribution rather than a fixed value. Combined with equation (1), it can be seen that the core of implementing the Bayesian network is to use sample data to find the posterior distribution of the model parameters, that is, p(ω|X, Y). At this point, the Bayesian theory can be directly used to obtain the following equation:

[0114]

[0115] Here p(ω) represents the prior distribution of the relevant parameters.

[0116] In the above equation, the numerator on the right side of the equation is relatively easy to solve. It consists of a likelihood function p(Y|X, ω) and a prior distribution p(ω). When it is converted into a log-likelihood function and maximized, the numerator is equivalent to obtaining the maximum a posteriori estimate with a weighted penalty term. However, in order to obtain the posterior distribution of the parameters in the Bayesian network, it is difficult to directly calculate formula (3) due to the existence of the denominator. Most of the current solutions to the posterior distribution of parameters rely on approximate methods, that is, setting a relatively easy variational distribution to fit the posterior distribution. In this case, the objective function can be rewritten as the following equation:

[0117]

[0118] Here, q θ (ω) is the variational distribution to be optimized, which is determined by the variational parameter θ.

[0119] From equation (4), we can see that we have transformed the unsolvable parameter posterior distribution problem into a variational distribution optimization problem, and used KL divergence to measure the two distributions so that the latter can continuously approach the posterior distribution of the model parameters under the current sample. During testing, we can use the learned variational distribution for sampling and effectively estimate uncertainty. At this time, formula (1) can be rewritten as the following equation:

[0120] p(y * |x * ,X,Y)≈∫p(y * |x * ,ω)q θ (ω)dω (5)

[0121] The method of using equation (5) to achieve prediction is called a variational Bayesian neural network, which is characterized by using variational distribution to approximate the difficult-to-obtain posterior distribution p(ω|X, Y). In order to obtain this variational distribution, it is generally achieved by minimizing equation (4). However, equation (4) also contains an integral step, and it is difficult to optimize it explicitly. Generally speaking, the Monte Carlo sampling method can be used to approximate the solution, and its loss function is:

[0122]

[0123] Here, n is the The total number of sampling times, ω i is the result of a certain sampling, L VI，i For this loss, τ is a preset hyperparameter.

[0124] The n in equation (6) is the number of samples, which means that the integral problem is converted into an expectation method for approximation. In addition, it can be found from the above equation that the first term on the right side of the equation corresponds to the likelihood function, indicating that the model should have sufficient data fitting ability, and the second term is to hope that the prior distribution of the parameters is close enough to the obtained variational distribution, which can be regarded as a regularization term. In order to balance the relationship between the two, the hyperparameter τ is introduced here, which can adjust the weights of the two more flexibly.

[0125] In the implementation of this method, formula (6) is equivalent to using the Monte Carlo method to transform a complex integral problem into a weighted sum problem. When optimization is required, the unbiased estimate of the original loss function gradient can be used to learn the distribution of neural network weights. This process looks like gradient backpropagation.

[16] Specifically, in the task of this method, in order to solve the silicon particle segmentation problem of the preheating metallographic image, its back propagation variational Bayes method is as follows:

[0126] Table 2 Back propagation Bayesian algorithm for aluminum alloy microstructure images

[0127]

[0128] In the pseudo code shown in Table 2, the UNet deep network is used as the basic model for the microscopic tissue image segmentation problem. At the same time, for the variational distribution, a Gaussian distribution with good properties is used, and the variational parameters at this time are the standard deviation σ and the mean μ. In order to obtain an effective numerical solution, the standard deviation is further converted into a variable ρ, and the Softplus function is used to ensure that its standard deviation is positive. Under this variational distribution, the distribution of all parameters in the deep network is determined by a pair of variational parameters θ = {ρ, μ}. In addition, for the prior distribution, this method considers two different forms, namely Gaussian distribution and Laplace distribution, which is equivalent to adding two-norm and one-norm penalty terms to the model parameters, which is conducive to avoiding overfitting of the model and improving its generalization ability.

[0129] A special case is that when the standard Gaussian distribution is selected as the parameter prior distribution and the Bernoulli distribution is used as the variational distribution for fitting, the optimization process of the model is equivalent to the traditional neural network with the introduction of random inactivation and regularization terms, and its loss function can be rewritten as:

[0130]

[0131] At this time, p is the probability of randomly dropping the layer.

[0132] The Bayesian method implemented based on equation (7) can also be regarded as the MC-Dropout method

[21] This is a relatively simple uncertainty estimation technique. Compared with Table 2, since the variational distribution here is based on the Bernoulli distribution with a fixed probability, the variational parameters that need to be optimized are consistent with the number of original network parameters, and are no longer twice the original model parameters. At the same time, the training process is the same as that of the neural network with the introduction of the random dropout layer, except that it is different during testing. Conventional neural networks will turn off the random dropout link during testing, while MC-Dropout requires multiple repeated predictions to capture uncertainty in the randomness of the model. Its training process is shown in Table 3.

[0133] Table 3 MC-Dropout algorithm for aluminum alloy microstructure image

[0134]

[0135] The design process of the uncertainty estimation method of the microstructure of the segmentation network model based on Beta distribution is as follows;

[0136] For the microstructure image analysis task considered by the present invention, based on the existing uncertainty estimation technology, the integration method and the Bayesian method have been implemented in the silicon particle segmentation problem, and a specific solution process has been given. However, such methods rely on the Monte Carlo approximation method in the prediction process, which means that more computing time is required. At the same time, for the integration method, it itself needs to store multiple deep models, which puts higher demands on computer storage space. These unfavorable factors limit the practical application of deep learning models.

[0137] Here, from the perspective of practicality, this paper proposes a novel uncertainty estimation method based on Beta distribution to improve the computational efficiency and practicality of the method. This is a method to achieve robust prediction by capturing the uncertainty of distribution. Inspired by the prior network [22-23] , the uncertainty in formula (1) can be rewritten as:

[0138]

[0139] Here, μ represents the parameter of the distribution, p(μ|x * , ω) represents the uncertainty of the distribution.

[0140] Equation (8) extends Equation (1) by introducing the distribution uncertainty p(μ|x * , ω), which becomes a double integral of the variable μ and the parameter ω. It can be seen that the uncertainty of the segmentation network model based on the Beta distribution will intervene in the distribution uncertainty p(μ|x * , ω), which in turn reflects the uncertainty of the data p(y * |μ). Such changes need to be designed according to actual problems, find a distribution form that can characterize the current data, and achieve robust prediction results.

[24] .

[0141] Specifically, the microstructure segmentation task involved in the present invention is essentially a pixel-level binary classification problem, where the foreground is silicon particles and the background is aluminum matrix and two interference items with similar appearances. Obviously, the first term p(y * |μ) should be a Bernoulli distribution, where the variable y * is binary, if it is a pixel where silicon particles are located, its value should be 1, otherwise it should be 0. For this distribution, its only parameter is the probability μ. A common method to describe the uncertainty of this distribution is to introduce its conjugate prior distribution, that is, to use the Beta distribution to realize the second term of the integral in formula (8), and its probability density distribution Beta (μ|α, β) is as follows:

[0142]

[0143] Among them, α and β are the two coefficients of Beta distribution, and the denominator Γ(·) represents the Gamma function.

[0144] Equation (9) here is the probability density function of the Beta distribution, which is used in the present invention to describe the uncertainty of the distribution. It is determined by two shape parameters. Once α and β are determined, the information about the expectation and variance of the Bernoulli distribution probability μ can also be obtained explicitly. Obviously, the prediction results obtained by the Beta distribution are more sufficient. Its expectation represents the probability of predicting silicon particles, and the variance can further reveal the confidence of the model in the prediction results of this sample, thereby correctly evaluating the uncertainty.

[0145] As for the third term p(ω|X, Y) in equation (8), we use it to describe the uncertainty of the parameters of the segmentation network model based on the Beta distribution. However, in the previous introduction, the posterior distribution of the parameters is generally difficult to be directly derived. If it is approximated according to the previous Bayesian variational method, it will also increase the inference time and reduce the prediction efficiency. Therefore, a special case is considered here, in which the prior distribution of the parameters is regarded as the Dirac distribution ω~δ(θ), that is, it is meaningful only when the variable ω=θ, and other cases are not considered. At this time, the posterior distribution of the parameters is directly equivalent to the prior distribution and has nothing to do with the data. The integral about dω in equation (8) will disappear, and its form can be simplified to:

[0146] p(y * |x * , X, Y) = ∫ p(y * |μ)p(μ|x * ,θ)dμ (10)

[0147] Equation (10) reveals the basic method of prediction of the segmentation network model based on Beta distribution. When there is a test sample x * When it is fed into the model, it can be seen that this is the integral of the Bernoulli distribution probability μ. Fortunately, since the distribution of μ adopts the form of Beta distribution, once the two shape parameters α and β are determined, its integral no longer requires a large number of samplings by the Monte Carlo method, but can directly obtain the closed-form solution of equation (10) through expectation, while evaluating the uncertainty, improving the prediction efficiency of the microstructure segmentation task, and the prediction results of the positive category (silicon particles) and the negative category (aluminum matrix and interference, etc.) are:

[0148]

[0149]

[0150] Theoretically, the segmentation network model based on Beta distribution proposed in the present invention uses Beta distribution to capture the uncertainty of the binary classification problem, simplifies the problem by using Dirac distribution ω~δ(θ), and uses a deep model to fit the origin coefficient θ in combination with real data, thereby achieving the purpose of estimating uncertainty in complex situations.

[0151] In the implementation process, in order to allow the neural network to learn the coefficient θ, it is necessary to set a reasonable loss function. The present invention adopts the maximum likelihood estimation method to achieve this. Specifically, consider any data sample in the training set Its maximum likelihood estimation equation for parameter θ can be expressed as:

[0152]

[0153] Its closed form is:

[0154]

[0155] Here, Bern(·) represents the Bernoulli distribution, and ψ(·) is the derivative of the logarithm of the Γ(·) function, called the digamma function.

[0156] In equation (13), the present invention uses the expected negative number of the log-likelihood function as the loss function of the model, and integrates the parameter μ in the Bernoulli distribution through mathematical derivation to obtain the expected closed form (14).

[0157] However, in the actual operation process, we found that relying solely on maximum likelihood estimation will lead to numerical instability, which is mainly caused by the scarcity of microscopic tissue image samples.

[0158] To this end, the present invention introduces a penalty term based on the truncation coefficient to add additional constraints to the model, as shown below:

[0159]

[0160] Among them, parameter C is a preset hyperparameter.

[0161] At this point, the loss function of the segmentation network model based on Beta distribution can be expressed as:

[0162] L Beta =L MLE (x,y;θ)+τL NormC (x,C;θ) (16)

[0163] Among them, the hyperparameter τ is used to balance the two loss functions.

[0164] Equation (16) shows the loss function of the Beta distribution segmentation network model in the present invention, which mainly includes the likelihood function L MLE (·) and the penalty term L NormC (·), the former is mainly used to fit the model to the data and obtain a closed-form solution through mathematical derivation, which is then integrated into the back-propagation process of the neural network, while the latter ensures the numerical stability of the optimization process through the truncation coefficient. The two work together to ensure that the model outputs a reasonable probability and effectively estimates the uncertainty. Its pseudo code is shown in Table 4 below:

[0165] Table 4 Training process of the segmentation network model based on Beta distribution for aluminum alloy microstructure images;

[0166]

[0167]

[0168] In Table 4, in order to ensure that the Beta distribution to be fitted is meaningful, the output result of the neural network is additionally added with the absolute value and exponential operation exp(|·|), so that the predicted shape coefficients α and β are both positive numbers greater than 1, and a more reasonable prediction result is obtained.

[0169] In order to realize the semantic segmentation method of silicon particles, the segmentation network model based on Beta distribution is adopted in the present invention. 9 ] network as a deep model. This is a classic deep convolutional neural network, whose name comes from the network structure that resembles the letter "U", which is specifically used to perform semantic segmentation tasks. The network was proposed by Ronneberger in 2015 and achieved excellent results in the ISBI cell tracking challenge in the same year. UNet and other networks with encoder-decoder structures (such as SegNet[ 25] ), the biggest innovation of UNet lies in its jump connection design. This design connects the feature maps of the same scale in the encoder and decoder, allowing the decoder to retain more detail information in the process of restoring the image size, thereby significantly enhancing the model's positioning ability and segmentation accuracy. The introduction of jump connections not only improves the final segmentation effect of the model, but also makes UNet occupy an important position in the field of semantic segmentation. Since the advent of UNet, many subsequent segmentation networks have borrowed this jump connection strategy. For example, Unet++[ 26 ]、HRNET[ 27 ] and Transformer-Unet[ 28] and other networks have been improved and expanded on this basis, playing a key role in many fields. In the present invention, considering the complexity of the silicon particle segmentation task, the segmentation network model based on Beta distribution is designed. Figure 5 shown.

[0170] Specifically, in order to show the designed UNet structure more intuitively and concisely, Figure 5 The batch normalization layer and random dropout layer in the UNet network are omitted. The UNet network contains feature maps of three scales, namely, the original image size of 376×376, the size of 188×188 after one downsampling, and the size of 94×94 after two downsampling. In the encoder part, as the scale is reduced, the number of features after the convolution operation gradually increases, which is 64, 128, and 128 respectively. Through this step-by-step downsampling method, the network can gradually extract more abstract and deep features of the input data. Subsequently, in the decoder part, these downsampled features are gradually upsampled by bilinear interpolation until the original image size is restored. After each upsampling, the jump connection design of UNet integrates the feature map of the corresponding encoder with the feature map of the decoder and performs a convolution operation. Finally, the result with the same size as the original image is output, which is converted into a probability value through the expectation of the Beta distribution, thereby realizing the segmentation task of silicon particles in the preheated metallographic image. In addition, when implementing the Bayesian network, the model parameters of this network will be expanded to twice that of the original network, corresponding to the two coefficients required to fit the variational distribution. When designing a segmentation network model based on Beta distribution, the output part of the model will remove the activation function and use the absolute value and exponential functions to fit the two shape parameters of the Beta distribution, thereby capturing the uncertainty of the original Bernoulli distribution.

[0171] The present invention uses the UNet network to segment the preheated image. In order to obtain a reasonable prediction probability, this application introduces three uncertainty estimation methods. For example, an integration method is used, which enhances the prediction ability of the model by introducing multiple models. In practice, 10 models are set up, and each model is trained after random initialization. There are two types of integration methods: deep integration and data set splitting. Deep integration is to train each UNet model with a complete data set, while data set splitting is to pre-divide the data set into 10 non-overlapping parts, and each model is trained with only a part of the data. Under the Adam optimizer with a learning rate of 1e-3, after 100 rounds of training, the integrated model was finally obtained. The advantage of this method is that by voting or averaging multiple models, the uncertainty of the prediction of a single model can be reduced, thereby improving the overall prediction accuracy.

[0172] For another example, for the variational Bayesian method, in the implementation process, the method uses two different prior distributions of model parameters: Gaussian distribution and Laplace distribution, the former has parameters μ=0, σ=0.1, and the latter has parameters μ=0, λ=0.1. In the model implementation, the back propagation Bayesian algorithm based on Gaussian variational posterior is used to optimize the variational parameters. In the training setting of the algorithm, the loss balance coefficient τ=1, the number of sampling during training is 4, and the number of Monte Carlo sampling during prediction is 100. In addition, in the MC-Dropout algorithm, different random dropout probabilities are set for different convolutional layers, where the random dropout probability of 3×3 convolutional layer is 0.1, and the random dropout probability of 1×1 convolutional layer is 0.5. Similar to the ensemble method, the two variational Bayesian algorithms also use the same optimizer and training details to ensure the fairness of the evaluation method.

[0173] For the segmentation network model based on Beta distribution proposed in the present invention, the key feature of the network is the introduction of a unique hyperparameter-truncation coefficient C, which is set to 2000 in practice. The training of the segmentation network model based on Beta distribution is similar to the integration method, and the same optimizer and number of training rounds are also used, but the loss balance coefficient τ is adjusted and set to 1e-3 to balance the weights of different loss terms. In addition, in order to increase the randomness in the model training process, a random deactivation layer with a probability of 0.1 is added to the backbone part. Under this parameter setting, the segmentation network model based on Beta distribution proposed in the present invention shows a better silicon particle prediction effect, and achieves better results in related indicators of uncertainty estimation. Combined with the good computational efficiency of the method itself, it fully verifies the effectiveness of the model in the task of aluminum alloy microstructure segmentation.

[0174] An aluminum alloy microstructure image segmentation device based on uncertainty estimation technology, comprising:

[0175] Acquisition module: used to acquire metallographic image samples of aluminum alloy after preheating treatment; to form a data set of metallographic image samples of aluminum alloy after preheating treatment;

[0176] Division module: used to divide the metallographic image sample data set after preheating treatment into a training set and a test set;

[0177] Enhancement module: used to enhance the aluminum alloy metallographic image sample data in the training set to obtain an enhanced aluminum alloy metallographic image training set;

[0178] Building module: used to build a segmentation network model based on Beta distribution for pixel-level segmentation of silicon particles in images;

[0179] Training module: Use the enhanced aluminum alloy metallographic image training set data to train the segmentation network model based on Beta distribution to obtain a trained segmentation network model based on Beta distribution;

[0180] Prediction module: Input the test set data into the trained segmentation network based on Beta distribution to predict the probability of silicon particles contained in the image.

[0181] A computer device comprises: a processor and a memory, wherein the memory stores a program module, and the program module runs on the processor to implement any one of the methods described above.

[0182] An aluminum alloy microstructure image segmentation device based on uncertainty estimation technology, comprising:

[0183] Memory;

[0184] processor; and computer program;

[0185] The computer program is stored in the memory and is configured to be executed by the processor to implement any one of the methods described.

[0186] The evaluation indicators used in this application;

[0187] In order to evaluate the method designed by the present invention, several evaluation indicators are introduced below. First, for the silicon particle segmentation problem, this is a pixel-level binary classification problem, and accuracy is a common evaluation indicator. In addition, considering the characteristics of the semantic segmentation task, in most cases, the intersection-over-union ratio and the Dice coefficient are more common evaluation indicators. The definitions of these three evaluation indicators are:

[0188]

[0189] Where: Acc represents the accuracy,

[0190]

[0191] Where: IoU means intersection over union;

[0192]

[0193] Among them, TP represents the number of positive category (silicon particles) pixels predicted correctly by the model, TN represents the number of negative category (background) pixels predicted correctly, and FP and FN are used to represent the number of positive and negative category pixels predicted incorrectly, respectively.

[0194] Secondly, considering that there are multiple interference items with similar appearances in the microscopic tissue images considered by the present invention, and some particles even require energy spectrum analysis to be accurately distinguished, it is necessary to determine whether the model can estimate its uncertainty. Ideally, the model can judge the confidence of its own prediction results, that is, it knows in advance whether the prediction results are accurate. In this case, experts can only perform a secondary manual analysis on the parts with high uncertainty, helping enterprises reduce labor costs.

[0195] In order to obtain the uncertainty of the prediction results, the present invention adopts the following two means:

[0196] MaxP=2·(1-max(Pred, 1-Pred)) (20)

[0197] Where: MaxP represents the maximum probability value;

[0198] NormEn=(Pred log(Pred)+(1-Pred)log(1-Pred)) / log(2) (21)

[0199] Among them: NormEn represents the entropy value;

[0200] The above equations (20) and (21) are common uncertainty estimation indicators:

[24] The closer the model prediction probability is to 0 or 1, the smaller its value is, indicating that the model is more confident about the prediction result at this time, otherwise manual intervention may be required. At the same time, in order to obtain quantitative results, AUROC and AUPR are used as evaluation indicators to examine the correlation between prediction accuracy and uncertainty, and then determine whether the model can correctly evaluate uncertainty.

[0201] Experimental results and analysis

[0202] First, the effectiveness of the method on the segmentation task is evaluated to verify the designed integration method.

[15] , Variational Bayesian Neural Network [16-17] The effect in microscopic tissue image analysis is compared with the proposed segmentation network model method based on Beta distribution. To ensure fairness, these methods use the same UNet structure. With the help of NVIDIA 3090 graphics card, after 100 rounds of training, the average results of the model after five-fold cross validation are shown in Table 5.

[0203] Table 5 Performance indicators of aluminum alloy microstructure image segmentation task

[0204] Acc IoU Dice Original UNet 0.9743 0.8283 0.9060 Original UNet+Dropout 0.9740 0.8233 0.9030 Deep Integration UNet 0.9748 0.8305 0.9074 Dataset Split UNet 0.9746 0.8294 0.9067 Backpropagation Bayesian (Gaussian distribution) 0.9747 0.8301 0.9071 Backpropagation Bayesian (Laplacian distribution) 0.9747 0.8297 0.9069 MC-Dropout 0.9745 0.8270 0.9053 Beta distribution segmentation network model 0.9749 0.8308 0.9075

[0205] In the table above, these methods are divided into four categories, namely, classic deep convolutional models, deep models based on ensemble learning, variational Bayesian networks, and the invented Beta distribution segmentation network model. On the right side of the table, the accuracy, IoU score, and Dice score are displayed simultaneously to analyze the effects of different methods in segmentation tasks.

[0206] Among them, the traditional deep network model part includes the original UNet network and the UNet model with the introduction of random dropout strategy. The experimental results show that the original UNet performs relatively average in the segmentation task, with an accuracy of 0.9743 and an IoU score of 0.8283. Although these indicators seem acceptable, they are slightly inferior to other deep convolutional networks. Considering the small number of microscopic tissue images, the model has an overfitting problem. After the introduction of the random dropout layer, this phenomenon did not improve. The accuracy and IoU score of the model dropped to 0.9740 and 0.8233 respectively, indicating that the introduction of random dropout alone failed to significantly improve the generalization ability of the model.

[0207] In the UNet deep model based on the ensemble method (the second group in Table 5), the invention studied two methods: deep ensemble and dataset splitting. The results show that the deep ensemble method performs slightly better than the latter, with accuracy rates of 0.9748 and 0.9746 respectively. This is different from the ensemble methods in traditional machine learning (such as random forests). In the ensemble methods of deep networks, the model trained with the complete dataset performs better than the dataset splitting method. The results are basically consistent with the results in the field of natural images.

[15] , indicating that for deep networks, random initialization through different seed points is sufficient to provide model diversity, thereby improving the overall performance of the ensemble method.

[0208] For the three variational Bayesian convolutional networks (the third group in Table 5), we first studied two back-propagation Bayesian networks based on Gaussian prior and Laplace prior. The results show that the effects of these two priors are very similar. The accuracy and IoU score of the Gaussian prior method are slightly higher than those of the Laplace prior method, but the difference is small, and both are significantly better than the original UNet network. This shows that the introduction of the variational Bayesian method has indeed improved the prediction performance of the model. In particular, the Dice coefficient of the MC-Dropout method has increased to 0.9053, which further verifies the effectiveness of Bayesian thinking in suppressing model overfitting.

[0209] Finally, by comparison, it can be seen that the Beta-distributed segmentation network model proposed in the invention performs best in the aluminum alloy microstructure image segmentation task, with an accuracy of 0.9749, an intersection-over-union ratio of 0.8308, and a Dice coefficient of 0.9075. Compared with other models, the advantage of the Beta-distributed segmentation network model is that it introduces the Beta distribution internally to better capture the uncertainty in the Bernoulli distribution. In addition, the Beta-distributed segmentation network model also uses the maximum likelihood loss function, which effectively corrects the prediction bias of the model and avoids the phenomenon of overconfidence in the original deep model. This improvement is reflected not only in the numerical indicators, but also in the model prediction results. In order to verify this, Figure 6 The prediction results of the original UNet network and the Beta distribution segmentation network model are shown respectively, where (a) is a metallographic image, (b) is the prediction result of the original UNet network, and (c) is the prediction result of the Beta distribution-based segmentation network model;

[0210] from Figure 6 It can be seen intuitively that the prediction results of the original UNet network show obvious "either 0 or 1" characteristics. This means that when the network deals with areas mixed with silicon particles and other interference items, it still gives highly confident predictions even if these areas are actually highly controversial. This phenomenon is particularly obvious in the enlarged image. The confidence of the original UNet in these complex areas is almost the same as those in the obvious silicon particle areas, which reflects its shortcomings in capturing uncertainty.

[0211] In sharp contrast, the Beta distribution segmentation network model we proposed shows more sophisticated and reasonable predictions when dealing with these controversial areas. The Beta distribution segmentation network model can give corresponding probability values ​​according to the degree of uncertainty in the region, rather than simply making a binary decision of "either 0 or 1". Such prediction results not only contain more information, but also make the model more reliable when dealing with complex scenes. This is also one of the key reasons why the Beta distribution segmentation network model can obtain higher scores in semantic segmentation tasks.

[0212] In order to further analyze the ability of different methods in capturing uncertainty, we further designed experiments to explore the relationship between misclassified samples and uncertainty. In order to reveal the differences in the uncertainty prediction capabilities of different models when dealing with complex and challenging segmentation tasks, we present the quantitative evaluation indicators and visualization results of uncertainty in Tables 6 and 7, respectively. Figure 7 , where (a) is the misclassified pixel map, (b) is the visualization of the segmentation network model (MaxP) based on the Beta distribution, and (c) is the visualization of the segmentation network model (NormEn) based on the Beta distribution.

[0213] Table 6 Uncertainty evaluation indicators for silicon particle segmentation problem

[0214]

[0215]

[0216] First, the experimental results of two different uncertainty estimation methods are shown in Table 6, namely, equation (20) based on maximum probability and equation (21) based on entropy. These methods reveal the model's ability to capture uncertainty of misclassified samples by calculating AUROC and AUPR (area under the precision-recall curve). Ideally, if a model can effectively capture uncertainty, then its uncertainty value will increase significantly in areas that are difficult for the model to identify, reflecting the model's ability to correct the predicted probability and its uncertainty perception ability.

[0217] In the results of Table 6, it can be clearly seen that the methods combined with uncertainty estimation significantly outperform the original UNet model and the UNet model with random dropout in both AUROC and AUPR. This means that these methods can better identify potential errors and adjust their confidence appropriately when dealing with complex or blurred image areas. It is particularly noteworthy that the MC-Dropout method performs outstandingly under the maximum probability-based strategy (MaxP), with an AUROC of 0.9720 and an AUPR of 0.9991. This shows that the MC-Dropout method has extremely high sensitivity and accuracy in predicting uncertainty. The segmentation network model based on Beta distribution proposed in the invention also shows excellent performance, especially in the AUROC indicator, with a score of 0.9721, the highest among all methods. This result proves that the segmentation network model based on Beta distribution not only has high accuracy in segmentation tasks, but also has a strong ability to handle uncertainty.

[0218] In addition, in Table 6, the performance of the two strategies, MaxP based on prediction value and NormEN based on entropy, is almost the same under different methods. Figure 6 In the visualization results, some subtle differences can be observed. Figure 7 Three metallographic images of preheat treatment taken at different holding times are shown to examine the performance of the segmentation network model based on Beta distribution in capturing uncertainty. Specifically, the first column is the misclassified pixel map (white pixels), the second column is the uncertainty estimation heat map based on the MaxP strategy, and the third column is the uncertainty estimation heat map based on the NormEN strategy.

[0219] from Figure 7From the heat map, we can see that the uncertainty estimation map under the NormEN strategy looks brighter than the heat map under the MaxP strategy, indicating that the uncertainty values ​​calculated by the NormEN strategy are generally high. However, in terms of actual performance indicators, the difference between the two is not obvious, and both show extremely high capture capabilities. More importantly, whether using the MaxP or NormEN strategy, the high uncertainty areas are mainly concentrated in two key locations: one is the boundary of the silicon particles, and the other is the area of ​​suspected interference. These areas usually have higher ambiguity or complexity, so it is reasonable for the model to give them higher uncertainty. This finding is highly consistent with the experts' thinking during the analysis, that is, in these fuzzy areas, the accuracy of the segmentation results may be questionable, and therefore show higher uncertainty values.

[0220] In general, these results fully verify the superiority of the Beta distribution segmentation network model in complex scenarios, indicating that it not only has an advantage in accuracy, but also can better handle and express the uncertainty in the prediction process. This makes the Beta distribution segmentation network model more valuable in practical applications, especially in scientific research and industrial testing that require high reliability and rigor. From the perspective of practical applications, if deep learning models are to be extended to more complex scenarios, simply pursuing accuracy can meet practical needs. Especially in key areas such as materials science microstructure testing, it is particularly important to evaluate the uncertainty of model prediction results. The model not only needs to have high accuracy, but also needs to be able to recognize and perceive its own familiarity with the data, and make cautious judgments when encountering unknown or ambiguous areas, so that experts can conduct secondary inspections. The Beta distribution segmentation network model shows its unique advantages in this regard.

[0221] Finally, the invention will examine the similarities and differences of several methods from the perspective of computational efficiency. Specifically, here we mainly analyze their space complexity and time complexity, which are important factors that directly affect the deployability and operating efficiency of the model. We will intuitively display them in Figure 8 middle.

[0222] Figure 8 A detailed analysis of the model capacity, computational efficiency, and segmentation effect of different image segmentation methods is presented. Model capacity is represented by a bar graph, showing the total number of parameters that need to be trained in each method. Time complexity, represented by a solid line graph, refers to the number of microscopic tissue images that the model can process per second (FPS), which directly affects the actual running speed and efficiency of the model. The segmentation effect is represented by a dotted line, which mainly uses the Dice coefficient to measure the segmentation accuracy of the model.

[0223] Overall, the methods have obvious differences in model capacity, computational efficiency, and segmentation effect. The number of parameters of the original UNet network is 214k. Although its segmentation effect is low, it performs well in speed and can achieve a processing speed of 17.22 frames per second. This makes the original UNet have a high processing efficiency in practical applications. However, for MC-Dropout, its test speed is significantly reduced to only 1.92FPS. This speed reduction is mainly attributed to the complex calculation process of the Monte Carlo sampling method. Although this process can improve the accuracy of model performance and uncertainty estimation, it also significantly increases the computational cost. The Bayesian network method also faces similar challenges. Due to its complex sampling operation, the computational speed is low, with FPS of 0.81 and 0.82 respectively, and the number of model parameters doubles to 429k, which also increases storage requirements and computational burden. In contrast, the deep integration method has the largest number of parameters among all methods, reaching 2.14M. This is because the total number of parameters will double with each additional base classifier. Although the test speeds of the deep integration and dataset splitting methods are higher than the variational Bayes method, 11.09FPS and 10.94FPS respectively, their computational and storage overheads are still large.

[0224] Among these methods, the proposed Beta distribution segmentation network model shows outstanding comprehensive performance. The network is almost the same as the original UNet in model capacity (214k) because it only sets the output channel to 2 at the very end to adapt to the two shape parameters α and β of the Beta distribution. Due to the characteristics of the Beta distribution, the prediction results of the Beta distribution segmentation network model can be calculated directly without repeated sampling or multiple iterations, thereby achieving a high efficiency of 17.02FPS in the test phase, which is close to the original UNet. This high efficiency, coupled with excellent segmentation effects and accurate uncertainty estimation, gives the Beta distribution segmentation network model significant advantages in practical applications. Its design not only optimizes computational efficiency, but also ensures the accuracy and practicality of the model when processing complex tasks, especially in actual scenarios that require fast response and high precision. Such a design can ensure that the model has stronger practical value and is convenient for deployment and implementation in actual scenarios.

[0225] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

[0226] [1]García-Moreno AI,Alvarado-Orozco J M,Ibarra-Medina J,et al.Image-based porosity classification in Al-alloys by laser metal deposition usingrandom forests[J].The International Journal of Advanced ManufacturingTechnology,2020,110:2827-2845.

[0227] [2]Aguiar J A,Gong M L,Unocic R R,et al.Decoding crystallography fromhigh-resolution electron imaging and diffraction datasets with deep learning[J].Science Advances,2019,5(10):eaaw1949.

[0228] [3]Müller S,Sauter C,Shunmugasundaram R,et al.Deep learning-basedsegmentation of lithium-ion battery microstructures enhanced by artificiallygenerated electrodes[J].Nature communications,2021,12(1):6205.

[0229] [4]Wijesinghe K,Wanni J,Banerjee N K,et al.Characterization ofmicroscopic deformation of materials using deep learning algorithms[J].Materials&Design,2021,208:109926.

[0230] [5]Ma B,Ban X,Su Y,et al.Fast-FineCut:Grain boundary detection inmicroscopic images considering 3D information[J].Micron,2019,116:5-14.

[0231] [6]Dutta T,Das D,Banerjee S,et al.An automated morphologicalclassification of ferrite-martensite dual-phase microstructures[J].Measurement,2019,137:595-603.

[0232] [7]Sarkar S S,Ansari M S,Mahanty A,et al.Microstructure imageclassification:Aclassifier combination approach using fuzzy integral measure[J].Integrating Materials and Manufacturing Innovation,2021,10(2):286-298.

[0233] [8]Breumier S,Ostormujof T M,Frincu B,et al.Leveraging EBSD data bydeep learning for bainite,ferrite and martensite segmentation[J].MaterialsCharacterization,2022,186:137805.

[0234] [9]Ronneberger O,Fischer P,Brox T.U-net:Convolutional networks forbiomedical image segmentation[C] / / Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015:18th International Conference,Munich,Germany,October 5-9,2015,Proceedings,Part III 18.Springer InternationalPublishing,2015:234-241.

[0235]

[10] He K,Gkioxari G,Dollar P,et al.Mask R-CNN[J].IEEE Transactions onPattern Analysis&Machine Intelligence,2020,42(02):386-397.

[0236]

[11] Okunev A G,Mashukov M Y,Nartova A V,et al.Nanoparticlerecognition on scanning probe microscopy images using computer vision anddeep learning[J].Nanomaterials,2020,10(7):1285.

[12] Chen Q,Zheng C,Cui Y,etal.A deep learning model for automatic analysis of cavities in irradiatedmaterials[J].Computational Materials Science,2023,221:112073.

[0237]

[13] Rahaman R.Uncertainty quantification and deep ensembles[J].Advances in Neural Information Processing Systems,2021,34:20063-20075.

[0238]

[14] Gawlikowski J,Tassi C R N,Ali M,et al.A survey of uncertainty indeep neural networks[J].Artificial Intelligence Review,2023,56(Suppl 1):1513-1589.

[0239]

[15] Lakshminarayanan B,Pritzel A,Blundell C.Simple and scalablepredictive uncertainty estimation using deep ensembles[J].Advances in NeuralInformation Processing Systems,2017,30.

[16] Blundell C,Cornebise J,KavukcuogluK,et al.Weight uncertainty in neural network[C] / / International Conference onMachine Learning.PMLR,2015:1613-1622.

[0240]

[17] Gal Y,Ghahramani Z.Dropout as a bayesian approximation:Representing model uncertainty in deep learning[C] / / International Conferenceon Machine Learning.PMLR,2016:1050-1059.

[0241]

[18] Breiman L.Random forests[J].Machine Learning,2001,45:5-32.

[0242]

[19] Cao Y,Geddes T A,Yang J Y H,et al.Ensemble deep learning inbioinformatics[J].Nature Machine Intelligence,2020,2(9):500-508.

[0243]

[20] D'Angelo F,Fortuin V.Repulsive deep ensembles are bayesian[J].Advances in Neural Information Processing Systems,2021,34:3451-3465.

[0244]

[21] Gal Y,Ghahramani Z.Bayesian convolutional neural networks withBernoulli approximate variational inference[J].arXiv preprint arXiv:1506.02158,2015.

[0245]

[22] Malinin A,Gales M.Predictive uncertainty estimation via priornetworks[J].Advances in Neural Information Processing Systems,2018,31.

[0246]

[23] Malinin A,Gales M.Reverse kl-divergence training of priornetworks:Improved uncertainty and adversarial robustness[J].Advances inNeural Information Processing Systems,2019,32.

[0247]

[24] Liang S,Li Y,Srikant R.Enhancing the reliability of out-of-distribution image detection in neural networks[J].arXiv preprint arXiv:1706.02690,2017.

[0248]

[25] Badrinarayanan V,Kendall A,Cipolla R.Segnet:A deep convolutionalencoder-decoder architecture for image segmentation[J].IEEE Transactions onPattern Analysis and Machine Intelligence,2017,39(12):2481-2495.

[0249]

[26] Zhou Z,Siddiquee M M R,Tajbakhsh N,et al.Unet++:Redesigning skipconnections to exploit multiscale features in image segmentation[J].IEEETransactions on Medical Imaging,2019,39(6):1856-1867.

[0250]

[27] Sun K,Xiao B,Liu D,et al.Deep high-resolution representationlearning for human pose estimation[C] / / Proceedings of the IEEE / CVF Conferenceon Computer Vision and Pattern Recognition.2019:5693-5703.

[0251]

[28] Sha Y,Zhang Y,Ji X,et al.Transformer-unet:Raw image processingwith unet[J].arXiv preprint arXiv:2109.08417,2021.

Claims

1. A method for segmenting aluminum alloy microstructure images based on uncertainty estimation technology, characterized in that: The following steps are involved: Acquire metallographic image samples of aluminum alloy after preheat treatment to form a metallographic image sample data set of aluminum alloy after preheat treatment; The sample data set of aluminum alloy metallographic images after preheat treatment is divided into a training set and a test set; The aluminum alloy metallographic image sample data in the training set is enhanced to obtain an enhanced aluminum alloy metallographic image sample training set; Construct a segmentation network model based on Beta distribution to perform pixel-level segmentation of silicon particles in images; The enhanced aluminum alloy metallographic image sample training set data is used to train the segmentation network model based on Beta distribution, and a trained segmentation network model based on Beta distribution is obtained; The test set data is input into the trained segmentation network based on Beta distribution to predict the probability of silicon particles contained in the image.

2. The aluminum alloy microstructure image segmentation method based on uncertainty estimation technology according to claim 1 is characterized in that: The process of constructing a segmentation network model based on Beta distribution for pixel-level segmentation of silicon particles in an image is as follows: The following formula is used to predict the uncertainty of aluminum alloy metallographic image samples: Here, μ represents the parameter of the distribution, p(μ|x * ,ω) represents the uncertainty of distribution; p(ω|X,Y) represents the uncertainty of model parameters; p(y*|x*,ω) represents the prediction result of the metallographic image x* of the tested aluminum alloy; Beta distribution is used to implement the second term of the integral in formula (8), and its probability distribution is used to describe the uncertainty of Bernoulli distribution. The probability density distribution is as follows: Among them, α and β are the two coefficients of Beta distribution, and the denominator Γ(·) represents the Gamma function; The prior distribution of parameter θ is regarded as Dirac distribution ω~δ(θ), that is, it is meaningful only when the variable ωω=θ. At this time, the posterior distribution of the parameter is directly equivalent to the prior distribution and has nothing to do with the data. The integral of dω in equation (8) will disappear, and equation (8) can be simplified to obtain the following equation: p(y * |x * ,X,Y)=∫p(y * |μ)p(μ|x * ,θ)dμ (10) When a metallographic sample x* of aluminum alloy is tested and sent to the segmentation network model based on Beta distribution, formula (10) is equivalent to the integral of the probability μ of Bernoulli distribution, where μ obeys Beta distribution. Once the two shape parameters α and β of Beta distribution are determined, the closed-form solution of equation (10) can be directly obtained through expectation. While evaluating the uncertainty, the prediction ability of microstructure segmentation task is improved. The positive category is silicon particles, and the negative category is aluminum matrix and interference. The prediction results are as follows:

3. The aluminum alloy microstructure image segmentation method based on uncertainty estimation technology according to claim 2 is characterized in that: The parameter θ of the segmentation network based on Beta distribution uses a unique loss function during the training process, and the maximum likelihood estimation equation of the parameter θ is L MLE (x, y; θ) is expressed as: Its closed form is: Here, Bern(·) represents Bernoulli distribution, and ψ(·) is the derivative of the logarithm of the Γ(·) function.

4. The aluminum alloy microstructure image segmentation method based on uncertainty estimation technology according to claim 1 is characterized in that: The segmentation network model based on Beta distribution designs additional constraints and adopts a penalty term L based on the truncation coefficient. NormC (x, C; θ) is used as a regularization factor, as shown below: Among them, parameter C is a preset hyperparameter.

5. The aluminum alloy microstructure image segmentation method based on uncertainty estimation technology according to claim 1 is characterized in that: The overall loss function of the segmentation network model based on Beta distribution is expressed as follows: L Beta =L MLE (x, y;θ)+τL NormC (x, C;θ) (16) Among them, the hyperparameter τ is used to balance the two loss functions, where the likelihood function L MLE (·) is used to fit the segmentation network model based on Beta distribution with supervised information, and obtain a closed-form solution through mathematical derivation, which is then integrated into the back propagation process of the neural network. NormC (x, C; θ) ensures the numerical stability of the optimization process through the truncation coefficient. The two work together to ensure that the model outputs a reasonable probability and effectively estimates the uncertainty.

6. The aluminum alloy microstructure image segmentation method based on uncertainty estimation technology according to claim 1 is characterized in that: The process of using the enhanced aluminum alloy metallographic image sample training set data to train the segmentation network model based on Beta distribution to obtain the trained segmentation network model based on Beta distribution is as follows: Obtain aluminum alloy microstructure image {X, Y}, truncation coefficient C, and balance coefficient τ; Randomly initialize the parameter θ; Collect sample-label pairs (x, y) from the aluminum alloy microstructure image set (X, Y); In the output stage of the Beta distribution segmentation network model, the output result after the terminal convolutional layer is transformed, that is, exp(|·|)=α,β; Using the closed form of the maximum likelihood estimation equation, calculate the maximum likelihood-based loss function L MLE (·); When the output α+β>C after the transformation of the segmentation network model based on the Beta distribution: use the penalty term L NormC (x, C; θ), introduce the penalty loss L NormC (·); Combine the loss function of the segmentation network model based on Beta distribution with the balance coefficient τ to calculate the overall loss L of the segmentation network model based on Beta distribution Beta ; The model parameter θ is updated according to the gradient of the segmentation network model based on Beta distribution to realize the training of the segmentation network model based on Beta distribution.

7. The aluminum alloy microstructure image segmentation method based on uncertainty estimation technology according to claim 1 is characterized in that: The segmentation network model based on Beta distribution is implemented by using an improved UNet deep convolutional network; The improved UNet deep convolutional network contains feature maps of three scales, namely, the original image size is 376×376, the size after one downsampling is 188×188, and the size after two downsampling is 94×94; In the encoder part, as the scale decreases, the number of features after the convolution operation gradually increases, which are 64, 128, and 128 respectively; through this gradual downsampling method, more abstract and deep features of the input data are gradually extracted; In the decoder part, the downsampled features are gradually upsampled by bilinear interpolation until they are restored to the original image size. After each upsampling, the jump connection design of UNet integrates the feature map of the corresponding encoder with the feature map of the decoder and performs a convolution operation. Finally, the result with the same size as the original image is output and converted into a probability value through the expectation of Beta distribution, thereby achieving the task of segmenting silicon particles in the preheating metallographic image.

8. An aluminum alloy microstructure image segmentation device based on uncertainty estimation technology, characterized in that: include: Acquisition module: used to acquire metallographic image samples of aluminum alloy after preheating treatment, forming a data set of metallographic image samples of aluminum alloy after preheating treatment; Partitioning module: used to divide the aluminum alloy metallographic image sample data set after preheating treatment into a training set and a test set; Enhancement module: used to enhance the aluminum alloy metallographic image sample data in the training set to obtain an enhanced aluminum alloy metallographic image training set; Building module: used to build a segmentation network model based on Beta distribution for pixel-level segmentation of silicon particles in images; Training module: Use the enhanced aluminum alloy metallographic image training set data to train the segmentation network model based on Beta distribution to obtain a trained segmentation network model based on Beta distribution; Prediction module: Input the test set data into the trained segmentation network based on Beta distribution to predict the probability of silicon particles contained in the image.

9. A computer device comprising: A processor and a memory, wherein the memory stores a program module, wherein the program module runs on the processor to implement the method according to any one of claims 1 to 7.

10. An aluminum alloy microstructure image segmentation device based on uncertainty estimation technology, characterized in that: include: Memory; processor; and computer programs; The computer program is stored in the memory and is configured to be executed by the processor to implement the method according to any one of claims 1 to 7.

Images