A metallographic image segmentation method, system, computer device and storage medium

Abstract

The application provides a metallographic image segmentation method, system, computer device and storage medium, and belongs to the technical field of computer vision and machine learning, and the method comprises the following steps: obtaining metallographic sample with label and without label; constructing a student network and a teacher network, and the teacher network is updated with a lag behind the student network parameters; in the training, the labeled sample is input into the student network to calculate a supervised loss, and the teacher network is input to establish a class confidence library; the unlabeled sample is input into the teacher network to generate an initial pseudo label, a dynamic threshold is used to screen reliable pseudo labels, and class perception resampling is performed based on the confidence library to obtain a final pseudo label; the unlabeled sample is input into the student network to calculate an unsupervised loss in combination with the final pseudo label; the student network parameters are updated based on the two losses, and the trained student network is used for metallographic image segmentation. The application can suppress pseudo label noise and relieve class imbalance under the condition of limited labeled data, and improve the segmentation precision.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and machine learning technology, and specifically relates to a metallographic image segmentation method, system, computer device and storage medium. Background Technology

[0002] Metallographic image segmentation is a fundamental task in metallographic analysis within materials science. Its purpose is to accurately identify and delineate different phases, grain boundaries, inclusions, and other structural regions from metallographic images, providing quantitative data for material performance evaluation, process optimization, and quality control. With the development of deep learning technology, methods based on image semantic segmentation models such as fully convolutional networks and U-shaped networks have been widely applied to metallographic image segmentation tasks, achieving significantly better results than traditional image processing methods.

[0003] Currently, deep learning-based metallographic image segmentation methods primarily rely on supervised training using a large number of pixel-level labeled metallographic image samples. However, pixel-level labeling of metallographic images is a highly specialized and time-consuming task, requiring materials scientists to delineate microstructures such as grain boundaries and phase boundaries pixel by pixel. This labeling cost is prohibitively high, especially in scenarios like industrial quality inspection where frequent model updates are necessary, making it difficult to obtain sufficient labeled data. To alleviate the dependence on large amounts of labeled data, researchers have attempted to employ a semi-supervised learning strategy, which utilizes both a small number of labeled samples and a large number of unlabeled samples for model training simultaneously.

[0004] However, effectively suppressing the interference of pseudo-label noise on model training and improving the model's learning ability for difficult and minority categories, thereby obtaining high-precision metallographic image segmentation results, under the condition of limited labeled data, is a technical challenge. Specifically: First, the pseudo-labels generated by the teacher network inevitably have prediction errors, especially in difficult regions such as blurred metallographic boundaries and similar gray levels of different phases. Directly using noisy pseudo-labels to train the student network leads to a decline in model performance. Furthermore, the number of samples of different categories in the metallographic dataset is often severely unbalanced; for example, the matrix phase occupies most of the area while precipitates or inclusions occupy only a few pixels, resulting in insufficient learning of minority categories and low segmentation accuracy. Second, the parameter update strategies of the teacher network and the ratio of supervised to unsupervised losses in existing methods are relatively fixed, making it difficult to adapt to changes in the model's learning state during training, thus limiting the efficiency of semi-supervised learning and the final segmentation accuracy. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a metallographic image segmentation method, system, computer device, and storage medium.

[0006] To achieve the above objectives, the present invention provides a metallographic image segmentation method, comprising: Obtain labeled target metallographic image samples and unlabeled target metallographic image samples.

[0007] Based on an image semantic segmentation network, student and teacher networks with identical structures but different initialization parameters are constructed. The teacher network does not participate in gradient backpropagation but only follows the student network's parameter updates using an exponential moving average method. In the training of the student network, each iteration includes: The following steps are performed: First, input the labeled metallographic image into the student network to calculate the supervised loss value. Then, input the labeled metallographic image into the teacher network, outputting the predicted probabilities of each category in the target metallographic image pixel-by-pixel, and establishing a category confidence library based on these predicted probabilities. Next, input the unlabeled metallographic image into the teacher network, outputting the predicted probability distribution of each category in the target metallographic image pixel-by-pixel, selecting the category corresponding to the highest predicted probability for a single pixel, and generating initial pseudo-labels pixel-by-pixel. Based on the predicted probability distribution and the initial pseudo-labels, a dynamic threshold strategy based on pixel loss is used to filter the initial pseudo-label pixels, retaining pixels with loss values ​​lower than the dynamic threshold to obtain a reliable pseudo-label set. Based on the category confidence library, the reliable pseudo-label set is subjected to category-aware random resampling to obtain the final resampled pseudo-labels. Finally, input the unlabeled metallographic image into the student network, and calculate the unsupervised loss value using the final pseudo-labels. Based on the supervised and unsupervised loss values, update the student network parameters.

[0008] After a single iteration, the teacher network is updated based on the parameters updated by the student network in this round; the training is repeated until the student network converges, and the trained student network is used to segment the target metallographic image.

[0009] Preferably, the step of establishing a category confidence database based on predicted probabilities specifically includes: after using the teacher network to make pixel-by-pixel predictions on the labeled metallographic image, calculating the average predicted probability of each category across all pixels, and storing the average predicted probability as the initial confidence of the category in the category confidence database; as the training rounds increase, the confidence value of each category is continuously updated using a moving average method, so that the category confidence database can dynamically reflect the current cognitive reliability of the teacher network for each category; the category of the target metallographic image includes all metallographic tissue types to be segmented in the target metallographic image.

[0010] Preferably, based on the predicted probability distribution and the initial pseudo-labels, a dynamic threshold strategy based on pixel loss is used to filter the initial pseudo-label pixels, retaining pixels with loss values ​​lower than the dynamic threshold to obtain a reliable pseudo-label set. Specifically, this includes: calculating the cross-entropy loss value between the initial pseudo-label and the predicted probability distribution for each pixel, and using the cross-entropy loss value as the predicted loss value of the pixel; statistically analyzing the cross-entropy loss value distribution of all pseudo-label pixels in each category, and using the loss value at a preset quantile position as the dynamic threshold for each category; retaining pixels with cross-entropy loss values ​​lower than the dynamic threshold, and the retained pixels and their pseudo-labels constitute a reliable pseudo-label set.

[0011] Preferably, the reliable pseudo-label set is subjected to category-aware random resampling based on the category confidence library to obtain the final resampled pseudo-labels. Specifically, this includes: calculating the resampling weight of each category according to the confidence value of each category in the category confidence library, wherein the category with lower confidence is assigned a higher resampling weight; then, according to the resampling weight, the pixels in the reliable pseudo-label set are randomly undersampled or oversampled, so that the proportion of samples of the category with lower confidence in the final pseudo-label set is increased, thereby generating the final pseudo-labels.

[0012] Preferably, the step of inputting the unlabeled metallographic image into the student network and calculating the unsupervised loss value in combination with the final pseudo-label specifically includes: calculating the cross-entropy loss corresponding to the pseudo-label category in the predicted probability distribution pixel by pixel, summing all cross-entropy loss values ​​and dividing by the total number of pixels corresponding to the final pseudo-label to obtain the unsupervised loss value.

[0013] Preferably, the step of inputting the unlabeled metallographic image into the student network and calculating the unsupervised loss value in conjunction with the final pseudo-labels further includes: performing strong enhancement processing on the unlabeled metallographic image; the strong enhancement processing includes one or more combinations of random rotation, random scaling, random cropping, color dithering, and Gaussian noise addition; before inputting the unlabeled metallographic image into the teacher network and outputting the predicted probability distribution of each category in the target metallographic image pixel by pixel, the step further includes: performing weak enhancement processing on the unlabeled metallographic image; the weak enhancement processing includes one or more combinations of random horizontal flipping, random vertical flipping, and random rotation.

[0014] Preferably, in the exponential moving average method, the parameter update of the teacher network decreases as the number of training iterations increases; the supervised loss value and the unsupervised loss value are combined into a total loss by averaging when updating the student network.

[0015] The present invention also provides a metallographic image segmentation system, comprising: The data acquisition module is used to acquire labeled and unlabeled target metallographic image samples.

[0016] The training module is used to construct student and teacher networks with identical structures but different initialization parameters based on an image semantic segmentation network. The teacher network does not participate in gradient backpropagation but only follows the student network's parameter updates using an exponential moving average method. In the training of the student network, each iteration includes: inputting a labeled metallographic image into the student network to calculate the supervised loss value; inputting the labeled metallographic image into the teacher network and outputting the predicted probabilities of each category in the target metallographic image pixel-by-pixel, and establishing a category confidence database based on the predicted probabilities; inputting an unlabeled metallographic image into the teacher network and outputting the predicted probabilities of each category in the target metallographic image pixel-by-pixel. The probability distribution is used to select the category corresponding to the maximum predicted probability of a single pixel, generating initial pseudo-labels pixel by pixel. Based on the predicted probability distribution and the initial pseudo-labels, a dynamic threshold strategy based on pixel loss is used to filter the initial pseudo-label pixels, retaining pixels with loss values ​​lower than the dynamic threshold to obtain a reliable pseudo-label set. Based on the category confidence library, the reliable pseudo-label set is subjected to category-aware random resampling to obtain the final pseudo-labels after resampling. The unlabeled metallographic image is input into the student network, and the unsupervised loss value is calculated in combination with the final pseudo-labels. Based on the supervised loss value and the unsupervised loss value, the student network parameters are updated.

[0017] The application module is used to update the teacher network based on the parameters updated by the student network in this round after a single iteration; iterative training continues until the student network converges, and the trained student network is used to segment the target metallographic image.

[0018] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in the metallographic image segmentation method.

[0019] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute any of the steps in the metallographic image segmentation method.

[0020] The metallographic image segmentation method provided by this invention has the following beneficial effects: By constructing a dual-network collaborative learning framework of a student network and a teacher network, this invention achieves efficient utilization of unlabeled metallographic images without relying on a large amount of pixel-level labeled data. Specifically, the teacher network generates initial pseudo-labels for unlabeled samples and introduces a dynamic threshold strategy based on pixel loss to automatically filter low-quality pseudo-labels according to the prediction reliability of each pixel, effectively suppressing the misleading influence of noisy pseudo-labels on the student network. Simultaneously, a category confidence library is constructed using labeled samples, and the filtered reliable pseudo-labels undergo category-aware random resampling, allowing the model to focus on a few categories with low confidence and difficult segmentation during training, alleviating the common class imbalance problem in metallographic data. Furthermore, the teacher network follows the student network parameters with lag updates using an exponential moving average method, ensuring the stability and consistency of pseudo-label generation. The combined effect of the above mechanisms enables this scheme to significantly improve the metallographic image segmentation model's resistance to noise and pseudo-labels, as well as its learning ability for difficult class samples, under the condition of limited labeled data. This results in high-precision and highly robust semi-supervised metallographic image segmentation results, solving the core problems of high labeling costs, large pseudo-label noise, and class imbalance in existing technologies. Attached Figure Description

[0021] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a metallographic image segmentation method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall framework of the semi-supervised image segmentation method provided in the embodiments of the present invention; Figure 3 This is a visualization comparison of the segmentation results of different methods in this invention on the UHCS dataset; wherein, Figure 3 (a) is the original SEM image; Figure 3 (b) is the actual labeled image; Figure 3 (c) is the prediction graph of the AD-MT model; Figure 3 (d) is the prediction graph of the Con2Net model; Figure 3 (e) is the prediction plot of the CA-MT model; Figure 3 (f) is the prediction graph of the SMAST model; Figure 4 This is a visualization comparison of the segmentation results of different methods in this invention on the MetalDAM dataset; wherein, Figure 4(a) is the original SEM image; Figure 4 (b) is the actual labeled image; Figure 4 (c) is the prediction graph of the AD-MT model; Figure 4 (d) is the prediction graph of the Con2Net model; Figure 4 (e) is the prediction plot of the CA-MT model; Figure 4 (f) is the prediction graph of the SMAST model; Figure 5 This is a visual comparison of the segmentation results of different methods in this invention on the CMMD dataset; wherein, Figure 5 (a) is the original SEM image; Figure 5 (b) is the actual labeled image; Figure 5 (c) is the prediction graph of the AD-MT model; Figure 5 (d) is the prediction graph of the Con2Net model; Figure 5 (e) is the prediction plot of the CA-MT model; Figure 5 (f) is the prediction graph of the SMAST model. Detailed Implementation

[0023] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0024] The method provided by this invention can be summarized as follows: Figure 2 In this model, labeled images are input into the student network (DINOv2DPT) to calculate supervised loss, while the same images are processed by the teacher network (DINOv2DPT) updated using exponential moving average (EMA) to maintain the class confidence library. Unlabeled images are input into the teacher network to generate pseudo-labels, which are then jointly optimized with the class confidence library using a pixel-based loss dynamic thresholding strategy (LBDT) to obtain high-quality pseudo-labels. These pseudo-labels are then input into the student network to calculate unsupervised consistency loss. The losses from both paths are used together for backpropagation to update the student network parameters, while the teacher network parameters are updated lagging behind the student network using EMA. This ultimately achieves effective utilization of unlabeled metallographic images and improves model accuracy.

[0025] Based on this, the present invention provides a metallographic image segmentation method, specifically a semi-supervised image segmentation method based on adaptive equalization sampling and dynamic loss thresholding. This method aims to address the problems of scarce labeled data, imbalanced class distribution, high pseudo-label noise, and insufficient learning of tail classes in existing semi-supervised segmentation methods for metallographic segmentation tasks, thereby significantly improving the segmentation accuracy and robustness of the model under limited labeling. Specifically, as follows... Figure 1As shown, it includes the following steps:

[0026] S1. Data Preparation and Preprocessing Obtain a metallographic image dataset, such as the UHCS dataset (containing 24 labeled images and 215 unlabeled images). Randomly partition a portion of the labeled images (e.g., 75%) as the training set, and the remaining labeled images as the test set. Perform preprocessing such as normalization and scaling on all images. Unlabeled images are used only for training.

[0027] S2. Network Architecture Construction and Initialization The core of this invention lies in a semi-supervised learning framework called SMAST, which comprises a student network and a teacher network. These are image semantic segmentation networks with identical structures but different initialization parameters. This framework introduces two collaborative loss-based dynamic thresholding strategies and adaptive equalization sampling. The student network updates its parameters through gradient descent, while the teacher network's parameters are historical exponential moving averages of the student network's parameters, resulting in more stable predictions. In this embodiment, the backbone encoders of both the teacher and student networks employ the visual base model DINOv2 to leverage its powerful visual feature extraction capabilities; the decoder uses a dense prediction transformer (DPT) for better dense pixel prediction.

[0028] A mean-based teacher framework is constructed. Both the teacher and student networks use DINOv2-Large as the encoder and DPT as the decoder. The teacher network parameters are initialized to be the same as those of the student network. The exponential moving average smoothing coefficient is set to 0.99. This smoothing coefficient is not a fixed value, but decreases as the number of training iterations increases. Specifically, let the total number of training iterations be T, and the smoothing coefficient α at iteration t be... t According to the linear decay formula α t =α max −(α max −α min )×t / T, where α max =0.9999, α min =0.99. Initialize the category confidence library Conf as an all-1 vector.

[0029] S3, Model Training Strategy The training process involves two data streams. For labeled data, both the student and teacher networks are simultaneously input. The student network computes a supervised loss, while the teacher network's predictions are used to dynamically maintain a "class confidence library," which records the model's average confidence for each class prediction. For unlabeled data, the teacher network first generates initial pseudo-labels, which are then filtered out using a loss-based dynamic thresholding strategy to remove unreliable pseudo-label pixels with high losses. Next, adaptive equalization sampling assigns higher sampling probabilities to classes with lower current prediction confidence based on the class confidence library, and class-aware resampling is performed on the threshold-filtered pseudo-label pixels. Finally, the student network is updated by combining the supervised loss and the unsupervised consistency loss calculated based on the resampled pseudo-labels.

[0030] S301, Loss-based dynamic threshold strategy This strategy is used to filter out reliable pseudo-labels, and the specific steps are as follows: (1) Warm-up phase: Train the model on a small amount of labeled data until initial convergence, and calculate the average prediction loss of the model on all labeled samples as the baseline threshold. .

[0031] (2) Selection phase: In subsequent training, dynamic thresholds The decays exponentially with increasing training iteration number t, and the calculation formula is: Where C is a fine-tuning constant slightly greater than 1 (e.g., 1.0001) to ensure that more samples are retained in the initial stage; γ is the decay coefficient, which controls the rate of threshold decrease; and s is the number of preheating rounds.

[0032] (3) Filtering: For each pixel of each unlabeled sample, calculate the cross-entropy loss between its predicted probability distribution and the pseudo-label. Only retain the loss values. The pixels involved are used in the unsupervised loss calculation. This method can dynamically adapt to changes in predictive ability during model training and more accurately filter out false labels.

[0033] S302, Adaptive Equalization Sampling Strategy This strategy is used to alleviate the class imbalance problem. Its core is a dynamically maintained class confidence library and a sampling rate calculation function based on the library.

[0034] The category confidence base is a vector of length C equal to the total number of categories. During training, whenever a batch of labeled data is processed, the average predicted probability for each category c is calculated using the teacher model's predictions and the true labels. And through the exponential moving average formula Update the confidence values ​​for the corresponding categories in the database. This database reflects the model's current overall grasp of each category.

[0035] For category c, its adaptive sampling rate The calculation formula is: in, This is called the confidence level; a higher value indicates greater uncertainty in the model's prediction of that class, and potentially worse performance. The formula normalizes the confidence level so that the sampling rate for the worst-performing class is 1. Hyperparameters (Usually set to 0.5) is used to smooth the distribution of sampling weights and prevent over-focusing on a few categories.

[0036] When calculating the unsupervised loss, instead of using all pixels that pass the threshold screening, random sampling is performed independently for each category based on the sampling rate mentioned above. For category c, pixels are randomly sampled from its set of valid pseudo-label pixels. =max(1, ( ) pixels are involved in the loss calculation. This forces the model to pay more attention to the categories of difficulties that the teacher network currently considers difficult during training.

[0037] S303, Specific Training Execution Process Set hyperparameters such as total number of training epochs, batch size, and optimizer. Dynamic threshold parameters are set as follows: decay coefficient γ = 2, warm-up epochs s = 1, and fine-tuning constant C = 1.0001. The smoothing coefficient β for adaptive equalization sampling is set to 0.5.

[0038] In each training iteration, the labeled batch is processed first: labeled images are input into the student network, and the cross-entropy supervised loss is calculated. Simultaneously, this batch is input into the teacher network, and the class confidence library is updated based on its predictions and the true labels. Then, the unlabeled batch is processed: unlabeled images are weakly augmented and input into the teacher network to generate initial pseudo-labels and prediction probabilities. Next, a dynamic loss threshold is calculated based on the current iteration number, and the cross-entropy loss between each pixel prediction and pseudo-label is calculated, generating a binary mask where pixels with loss values ​​below the dynamic threshold are marked as 1, otherwise 0. Subsequently, the sampling rate for all classes in the current batch is calculated based on the latest updated class confidence library, and the final sampling mask is generated by combining the pseudo-labels, threshold mask, and sampling rate. Then, the cross-entropy loss between the student network's prediction of the strongly augmented version of the unlabeled image and the pseudo-label is calculated. Only the loss at the pixel positions marked as 1 in the sampling mask is retained and averaged to obtain the unsupervised consistency loss. Finally, the supervised and unsupervised losses are averaged and combined to obtain the total loss, L. total =(L sup +L unsupThe student network parameters are updated via backpropagation, and then the teacher network parameters are updated using an exponential moving average. Strong enhancement processing includes one or more combinations of random rotation, random scaling, random cropping, color dithering, and Gaussian noise addition; weak enhancement processing includes one or more combinations of random horizontal flipping, random vertical flipping, and random rotation.

[0039] Alternatively, a category-level quantile dynamic threshold can be used: the cross-entropy loss value distribution of all pseudo-label pixels in each category is statistically analyzed, and the loss value at a preset quantile (e.g., median 0.5) is used as the dynamic threshold for that category. Pixels with loss values ​​below this threshold are retained to form a reliable set of pseudo-labels. This method can also achieve the purpose of filtering reliable pseudo-labels.

[0040] S4. Model Evaluation and Application After training, the best-performing model is saved as the final model. Model performance is evaluated on the test set using metrics such as MIoU (Missing Information Unit), Accuracy, and the harmonic mean F1-score (F1 score) of precision and recall. Inputting a new metallographic image to be segmented into the model yields pixel-level semantic segmentation results. As shown in Table 1, the method of this invention (SMAST) outperforms other algorithms in segmentation performance with different annotation ratios. The AD-MT, Con2Net, and CA-MT models are existing models, and they are used as comparative methods against the SMAST framework of this invention. Figure 3 , Figure 4 , Figure 5 As shown in the figures, (a) is the original SEM image, (b) is the ground truth labeled image, (c) is the AD-MT model prediction image, (d) is the Con2Net model prediction image, (e) is the CA-MT model prediction image, and (f) is the SMAST model prediction image of this invention. The comparison shows that the segmentation results of the method of this invention are superior to the comparative methods in terms of boundary sharpness, small target integrity, and tail category recognition. Figure 3 , Figure 4 , Figure 5 The experimental results correspond to the UHCS, MetalDAM, and CMMD datasets, respectively.

[0041] Table 1. Segmentation performance of different methods on metallographic datasets Compared with the prior art, the present invention has the following advantages: (1) Significantly improved segmentation performance: Experiments on multiple public metallographic image datasets (UHCS, MetalDAM) and a self-built dataset (CMMD) show that the method of this invention consistently outperforms existing mainstream semi-supervised metallographic segmentation methods (such as AD-MT, Con2Net, CA-MT) in core metrics such as accuracy (Acc), F1 score, and mean intersection-over-union ratio (MIoU) at three annotation ratios of 25%, 50%, and 100%. For example, the MIoU reaches 77.42% on the UHCS dataset with 100% labeling and 72.70% on the MetalDAM dataset.

[0042] (2) Efficient use of unlabeled data: Through a loss-based dynamic threshold strategy, high-quality pseudo-labels can be screened more intelligently and precisely, reducing the interference of noisy labels on model training, and making unlabeled data more effectively utilized, especially with a more obvious performance advantage under low label ratio.

[0043] (3) Effectively alleviate class imbalance: Through an adaptive equalization sampling strategy, more training weights are dynamically allocated to tail categories that are difficult to learn, which significantly improves the model's segmentation integrity and accuracy for tail categories, such as small defects and rare phases in metallographic images.

[0044] (4) Enhance model robustness: The two modules work together to improve the model’s generalization ability and robustness in complex scenarios with scarce annotations and class imbalance, providing an effective solution for the automated analysis of metallographic images and other medical and remote sensing images with similar characteristics.

[0045] In summary, this invention addresses three core problems in semi-supervised segmentation of metallographic images: high pseudo-label noise, severe class imbalance, and fixed training strategies that are difficult to adapt to changes in model learning states. It proposes a semi-supervised learning framework called SMAST. To solve the pseudo-label noise problem, this invention designs a class-level quantile dynamic thresholding strategy based on pixel loss. By statistically analyzing the cross-entropy loss distribution of all pseudo-label pixels in each class, and using the loss value at a preset quantile position as a dynamic threshold, only reliable pixels with loss values ​​below this threshold are retained. This effectively suppresses the misleading influence of noisy pseudo-labels generated by the teacher network on the student network.

[0046] To address the class imbalance problem, a class confidence database is constructed using labeled samples to record the model's average prediction probability for each class. Based on this, classes with lower confidence are assigned higher resampling weights. Class-aware undersampling or oversampling is performed on the reliable pseudo-label set, forcing the model to focus on a few classes with low confidence and difficult segmentation (such as minute defects or rare phases in metallographic images) during training, significantly alleviating the sample size imbalance between the matrix phase and the precipitated phase. To address the fixed training strategy problem, an exponential moving average smoothing coefficient is adopted, which decays linearly with the number of iterations (from 0.9999 to 0.99). This ensures that the teacher network updates slowly in the early stages of training to provide stable pseudo-labels, and then quickly follows the progress of the student network in the later stages. Simultaneously, supervised and unsupervised losses are combined using an average summation method to balance the contributions of labeled and unlabeled data.

[0047] The synergistic effect of the above mechanisms enables this invention to achieve significant technical results under the condition of limited labeled data: Experiments on three metallographic datasets, UHCS, MetalDAM, and CMMD, show that when using only 25% of the labeled data, the average intersection-over-union (MIoU) of this invention is improved by 18.26%, 10.93%, and 8.70% respectively compared with the existing best comparison methods, and the F1 score is also significantly improved. Moreover, the segmentation results are superior to mainstream semi-supervised segmentation methods such as AD-MT, Con2Net, and CA-MT in terms of boundary clarity, small target integrity, and tail category recognition, which proves its superiority and robustness in pseudo-label noise suppression, class balance learning, and adaptive training strategy.

[0048] Based on the same inventive concept, the present invention also provides a metallographic image segmentation system, comprising: The data acquisition module is used to acquire labeled and unlabeled target metallographic image samples.

[0049] The training module is used to construct student and teacher networks with identical structures but different initialization parameters based on an image semantic segmentation network. The teacher network does not participate in gradient backpropagation but only follows the student network's parameter updates using an exponential moving average method. In the training of the student network, each iteration includes: inputting a labeled metallographic image into the student network to calculate the supervised loss value; inputting the labeled metallographic image into the teacher network and outputting the predicted probabilities of each category in the target metallographic image pixel-by-pixel, and establishing a category confidence database based on the predicted probabilities; inputting an unlabeled metallographic image into the teacher network and outputting the predicted probabilities of each category in the target metallographic image pixel-by-pixel. The probability distribution is used to select the category corresponding to the maximum predicted probability of a single pixel, generating initial pseudo-labels pixel by pixel. Based on the predicted probability distribution and the initial pseudo-labels, a dynamic threshold strategy based on pixel loss is used to filter the initial pseudo-label pixels, retaining pixels with loss values ​​lower than the dynamic threshold to obtain a reliable pseudo-label set. Based on the category confidence library, the reliable pseudo-label set is subjected to category-aware random resampling to obtain the final pseudo-labels after resampling. The unlabeled metallographic image is input into the student network, and the unsupervised loss value is calculated in combination with the final pseudo-labels. Based on the supervised loss value and the unsupervised loss value, the student network parameters are updated.

[0050] The application module is used to update the teacher network based on the parameters updated by the student network in this round after a single iteration; iterative training continues until the student network converges, and the trained student network is used to segment the target metallographic image.

[0051] This invention also provides a computer device. At the hardware level, the computer device includes a processor, an internal bus, a network interface, memory, and non-volatile memory, and may also include other hardware required for business operations. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the metallographic image segmentation method provided above.

[0052] The present invention also provides a computer-readable storage medium storing a computer program that can be used to execute the metallographic image segmentation method provided above.

[0053] Specific limitations regarding the computational system for metallographic image segmentation can be found in the limitations of the metallographic image segmentation method described above, and will not be repeated here. Each module in the aforementioned metallographic image segmentation system can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0054] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification. Furthermore, the above embodiments only illustrate several implementation methods of this application, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A metallographic image segmentation method, characterized in that, Includes the following steps: Obtain labeled and unlabeled target metallographic image samples; Based on an image semantic segmentation network, student and teacher networks with identical structures but different initialization parameters are constructed. The teacher network does not participate in gradient backpropagation but only follows the student network's parameter updates using an exponential moving average method. In the training of the student network, each iteration includes: The process involves several steps: First, inputting a labeled metallographic image into the student network to calculate the supervised loss. Then, inputting the labeled metallographic image into the teacher network, which outputs the predicted probabilities of each category in the target metallographic image pixel-by-pixel, and establishes a category confidence library based on these probabilities. Next, inputting an unlabeled metallographic image into the teacher network, which outputs the predicted probability distribution of each category in the target metallographic image pixel-by-pixel, selects the category corresponding to the highest predicted probability for a single pixel, and generates initial pseudo-labels pixel-by-pixel. Based on the predicted probability distribution and the initial pseudo-labels, a dynamic threshold strategy based on pixel loss is used to filter the initial pseudo-label pixels, retaining pixels with loss values ​​lower than the dynamic threshold to obtain a reliable pseudo-label set. Based on the category confidence library, the reliable pseudo-label set is subjected to category-aware random resampling to obtain the final resampled pseudo-labels. Finally, the unlabeled metallographic image is input into the student network, and the unsupervised loss value is calculated using the final pseudo-labels. Based on the supervised and unsupervised loss values, the student network parameters are updated. After a single iteration, the teacher network is updated based on the parameters updated by the student network in this round; the training is repeated until the student network converges, and the trained student network is used to segment the target metallographic image.

2. The metallographic image segmentation method according to claim 1, characterized in that, The establishment of a category confidence database based on predicted probabilities specifically includes: after using the teacher network to predict the labeled metallographic image pixel by pixel, calculating the average predicted probability of each category across all pixels, and storing the average predicted probability as the initial confidence of the category in the category confidence database; as the training rounds increase, the confidence value of each category is continuously updated using a moving average method, so that the category confidence database can dynamically reflect the current reliability of the teacher network's understanding of each category; the category of the target metallographic image includes all metallographic tissue types to be segmented in the target metallographic image.

3. The metallographic image segmentation method according to claim 1, characterized in that, Based on the predicted probability distribution and the initial pseudo-labels, a dynamic threshold strategy based on pixel loss is used to filter the initial pseudo-label pixels, retaining pixels with loss values ​​lower than the dynamic threshold to obtain a reliable pseudo-label set. Specifically, this includes: calculating the cross-entropy loss value between the initial pseudo-label and the predicted probability distribution for each pixel, and using the cross-entropy loss value as the predicted loss value of the pixel; statistically analyzing the cross-entropy loss value distribution of all pseudo-label pixels in each category, and using the loss value at a preset quantile position as the dynamic threshold for each category; retaining pixels with cross-entropy loss values ​​lower than the dynamic threshold, and the retained pixels and their pseudo-labels constitute a reliable pseudo-label set.

4. The metallographic image segmentation method according to claim 1, characterized in that, Based on the category confidence library, category-aware random resampling is performed on the reliable pseudo-label set to obtain the final resampled pseudo-label. Specifically, this includes: calculating the resampling weight for each category according to the confidence value of each category in the category confidence library, wherein the category with the lower confidence is assigned a higher resampling weight; then, according to the resampling weight, the pixels in the reliable pseudo-label set are randomly undersampled or oversampled, so that the proportion of samples of the category with the lower confidence in the final pseudo-label set is increased, thereby generating the final pseudo-label.

5. The metallographic image segmentation method according to claim 1, characterized in that, The step of inputting the unlabeled metallographic image into the student network and calculating the unsupervised loss value in combination with the final pseudo-label specifically includes: calculating the cross-entropy loss corresponding to the pseudo-label category in the predicted probability distribution pixel by pixel, summing all cross-entropy loss values ​​and dividing by the total number of pixels corresponding to the final pseudo-label to obtain the unsupervised loss value.

6. The metallographic image segmentation method according to claim 1, characterized in that, Before inputting the unlabeled metallographic image into the student network and calculating the unsupervised loss value in conjunction with the final pseudo-label, the method further includes: performing strong enhancement processing on the unlabeled metallographic image; the strong enhancement processing includes one or more combinations of random rotation, random scaling, random cropping, color dithering, and Gaussian noise addition; before inputting the unlabeled metallographic image into the teacher network and outputting the predicted probability distribution of each category in the target metallographic image pixel by pixel, the method further includes: performing weak enhancement processing on the unlabeled metallographic image; the weak enhancement processing includes one or more combinations of random horizontal flipping, random vertical flipping, and random rotation.

7. The metallographic image segmentation method according to claim 1, characterized in that, In the exponential moving average method, the parameter updates of the teacher network decrease as the number of training iterations increases; the supervised loss value and the unsupervised loss value are combined into a total loss by averaging when updating the student network.

8. A metallographic image segmentation system, characterized in that, include: The data acquisition module is used to acquire labeled and unlabeled target metallographic image samples. The training module is used to construct student and teacher networks with identical structures but different initialization parameters based on an image semantic segmentation network. The teacher network does not participate in gradient backpropagation but only follows the student network's parameter updates using an exponential moving average method. In the training of the student network, each iteration includes: inputting a labeled metallographic image into the student network to calculate the supervised loss value; inputting the labeled metallographic image into the teacher network and outputting the predicted probabilities of each category in the target metallographic image pixel-by-pixel, and establishing a category confidence database based on the predicted probabilities; inputting an unlabeled metallographic image into the teacher network and outputting the predicted probabilities of each category in the target metallographic image pixel-by-pixel. The probability distribution is used to select the category corresponding to the maximum predicted probability of a single pixel, generating initial pseudo-labels pixel by pixel. Based on the predicted probability distribution and the initial pseudo-labels, a dynamic threshold strategy based on pixel loss is used to filter the initial pseudo-label pixels, retaining pixels with loss values ​​lower than the dynamic threshold to obtain a reliable pseudo-label set. The reliable pseudo-label set is then subjected to category-aware random resampling based on the category confidence library to obtain the final pseudo-labels after resampling. The unlabeled metallographic image is input into the student network, and the unsupervised loss value is calculated in combination with the final pseudo-labels. The student network parameters are updated based on the supervised loss value and the unsupervised loss value. The application module is used to update the teacher network based on the parameters updated by the student network in this round after a single iteration; iterative training continues until the student network converges, and the trained student network is used to segment the target metallographic image.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to perform the steps of the method according to any one of claims 1 to 7.

Images