A MURA defect segmentation method, system, device and storage medium

Abstract

The present application discloses a MURA defect segmentation method, system, device, and storage medium, which are used to reduce the dependence on labels and improve the accuracy of MURA defect segmentation. The method of the present application includes: obtaining sample image data; training a classification network and a segmentation network according to the sample image data; obtaining the image to be segmented and inputting it into the classification network to obtain an initial CAM map; performing weighted fusion on all feature layers to obtain a binary refined CAM map and inputting it into the segmentation network to obtain a target CAM map; determining low-confidence regions in the target CAM map; segmenting the low-confidence regions to obtain refined regions and randomly copying them to normal regions to generate multiple prediction results, analyzing the consistency, and outputting the MURA defect status of the refined regions; when the MURA defect status indicates the existence of MURA defects, changing the refined regions to high-confidence regions; using the high-confidence regions as segmentation labels to perform MURA defect segmentation on the image to be segmented.

Description

Technical Field

[0001] The present application relates to the field of screen detection, and particularly to a method, system, device and storage medium for MURA defect segmentation. Background Art

[0002] In the field of display panel manufacturing, the MURA detection process is an important technological step. MURA defects usually refer to problems such as color difference, uneven brightness, defects, and uneven lighting in the display panel. These problems may affect the display effect of the panel and thus affect the product quality.

[0003] In a display panel, MURA defects usually appear as blurred and irregular regions, which makes it difficult to obtain high-quality segmentation labels by traditional annotation methods. In the prior art, MURA detection is mainly performed through threshold-based segmentation methods and region growing and watershed algorithms. The threshold-based segmentation method mainly extracts the defect region by setting thresholds for brightness, contrast, color, etc. It is simple but has poor effects on complex display panel problems (such as uneven brightness or minute defects). The region growing and watershed algorithm divides regions according to the pixel similarity in the image and is suitable for the detection of some continuous region defects. However, it is prone to over-segmentation in the presence of noise or complex backgrounds. Both of the above methods have problems of dataset imbalance and difficult annotation. In the display panel defect segmentation task, the annotation of the dataset is usually very difficult and resource-intensive. The defects may be local, minute, and of various types, which may lead to subjective differences among annotators during the annotation process. Moreover, the types and severity of the defects vary greatly, resulting in dataset imbalance and thus affecting the training effect of the model. Summary of the Invention

[0004] To solve the above technical problems, the present application provides a method, system, device and storage medium for MURA defect segmentation, which is used to reduce the dependence on labels and improve the accuracy of MURA defect segmentation.

[0005] The technical solutions provided in the present application are described below:

[0006] The first aspect of the present application provides a method for MURA defect segmentation, including:

[0007] Obtaining sample image data, where the sample image data is image data with pixel-level annotation for MURA defect segmentation;

[0008] Training a classification network and a segmentation network according to the sample image data, where the classification network is used to classify whether an image contains MURA defects, and the segmentation network is used to segment the MURA defect region;

[0009] Obtain the image to be segmented, and input the image to be segmented into the classification network to obtain an initial CAM map;

[0010] Extract all feature layers of the initial CAM map, and perform weighted fusion on all the feature layers to obtain a binarized refined CAM map;

[0011] Input the binarized refined CAM map into the segmentation network to obtain a target CAM map;

[0012] Determine the low-confidence regions in the target CAM map;

[0013] Segment the low-confidence regions to obtain refined regions;

[0014] Randomly copy the refined regions to normal regions, and generate multiple prediction results through the MC-DropOut technique;

[0015] Analyze the consistency of the multiple prediction results, and output the MURA defect status of the refined regions;

[0016] When the MURA defect status is that there are MURA defects, change the refined regions to high-confidence regions;

[0017] Use the high-confidence regions as segmentation labels to perform MURA defect segmentation on the image to be segmented.

[0018] Optionally, the training of the classification network according to the sample image data includes:

[0019] Initialize an initial classification network according to preset training parameters, and define the loss function of the initial classification network as a cross-entropy loss function;

[0020] Input the sample image data into the initial classification network one by one to obtain the MURA defect prediction probability, which is used to predict whether there are MURA defects in the sample image data;

[0021] Calculate the classification loss value between the MURA defect prediction probability and the true label through the cross-entropy loss function. The true label is obtained from the sample image data, and the cross-entropy loss function is as follows:

[0022] ;

[0023] where y i c is the classification label of the i-th image of the label classification sample, and p i cis the MURA defect probability value predicted by the classification network for the i-th image in the label classification sample, and N represents the number of images in the label classification sample;

[0024] Backpropagate the classification loss value to optimize the initial classification network, and input the next sample image data into the initial classification network until all the images included in the sample image data are input and the training is completed to obtain the classification network.

[0025] Optionally, extracting all the feature layers of the initial CAM map and performing weighted fusion on all the feature layers to obtain a binary fine CAM map includes:

[0026] Extract all the feature layers of the initial CAM map;

[0027] Determine the high-level feature map and the low-level feature map from all the feature layers;

[0028] Perform weighted fusion on the high-level feature map and the low-level feature map through the following formula to obtain a binary fine CAM map:

[0029] (x)；

[0030] where L is the total number of layers of all the feature layers, (x) is the activation value of the c-th channel when the layer number of the feature layer is 1, is the weight of the c-th channel when the layer number of the feature layer is 1, is the weighting coefficient of the layer.

[0031] Optionally, training the segmentation network according to the sample image data includes:

[0032] Construct an initial segmentation network with the U-Net network as the backbone network, and define the loss function and the defect area weighting coefficient of the initial segmentation network;

[0033] Relabel the MURA defect segmentation area of the sample image data with inaccurate labels to obtain pseudo-label image data, where the labeled area of the pseudo-label image is the foreground area and the unlabeled area is the background area;

[0034] Train the initial segmentation network with the pseudo-label image data and the sample image data, and calculate the loss of the MURA defect segmentation area of the pseudo-label image data and the sample image data through a weak supervision loss function to obtain a first segmentation loss value. The weak supervision loss function is as follows:

[0035] ；

[0036] Among them, is the first segmentation loss value, is the defect area weighting coefficient, where fg represents the foreground area and bg represents the background area, is the prediction value of the i-th image segmentation network of the pseudo-label image data and the sample image data;

[0037] By calculating the constraint loss function through the initial segmentation network, a second segmentation loss value is obtained. The constraint loss function is as follows:

[0038] ;

[0039] Among them, is the second segmentation loss value, is the refined CAM map corresponding to the i-th image data of the pseudo-label image data and the sample image data, i and is the prediction value of the i-th image segmentation network of the pseudo-label image data and the sample image data. i

[0040] Update the initial segmentation network through the first segmentation loss value and the second segmentation loss value until all the image data in the pseudo-label image data and the sample image data are completed for training, and a segmentation network is obtained.

[0041] Optionally, the updating the initial segmentation network through the first segmentation loss value and the second segmentation loss value includes:

[0042] Calculate the total loss value through the first segmentation loss value and the second segmentation loss value. The total loss value is calculated by the following formula:

[0043] ;

[0044] Among them, is the total loss value, , are the weight coefficients of the first segmentation loss value and the second segmentation loss value respectively ;

[0045] Update the initial segmentation network according to the total loss value.

[0046] Optionally, the segmentation network includes a foreground network and a background network, and the defect area weighting coefficient of the foreground network is greater than that of the background network.

[0047] ​Optionally, determining the low-confidence region in the target CAM map includes:

[0048] Obtaining a first confidence level of the target CAM map through a foreground network;

[0049] Obtaining a second confidence level of the target CAM map through a background network;

[0050] When the confidence levels of the first confidence level and the second confidence level are different, determining the confidence level of the target CAM map as a low confidence level;

[0051] When the confidence levels of the first confidence level and the second confidence level are the same, determining the confidence level of the target CAM map as a high confidence level.

[0052] A second aspect of the present application provides a MURA defect segmentation system, including:

[0053] A first acquisition unit, configured to acquire sample image data, where the sample image data is image data with pixel-level annotation for MURA defect segmentation;

[0054] A training unit, configured to train a classification network and a segmentation network according to the sample image data, where the classification network is used to classify whether an image contains MURA defects, and the segmentation network is used to segment the MURA defect region;

[0055] A second acquisition unit, configured to acquire an image to be segmented, and input the image to be segmented into the classification network to obtain an initial CAM map;

[0056] A weighted fusion unit, configured to extract all feature layers of the initial CAM map, and perform weighted fusion on all the feature layers to obtain a binary refined CAM map;

[0057] An input unit, configured to input the binary refined CAM map into the segmentation network to obtain a target CAM map;

[0058] A determination unit, configured to determine the low-confidence region in the target CAM map;

[0059] A first segmentation unit, configured to segment the low-confidence region to obtain a refined region;

[0060] A prediction unit, configured to randomly copy the refined region to a normal region, and generate multiple prediction results through the MC-DropOut technique;

[0061] An analysis unit, configured to analyze the consistency of the multiple prediction results, and output the MURA defect state of the refined region;

[0062] A change unit, configured to change the refined region into a high-confidence region when the MURA defect status indicates the existence of a MURA defect;

[0063] A second segmentation unit, configured to perform MURA defect segmentation on the image to be segmented using the high-confidence region as a segmentation label.

[0064] Optionally, the training unit is specifically configured to:

[0065] Initialize an initial classification network according to preset training parameters, and define the loss function of the initial classification network as a cross-entropy loss function;

[0066] Input the sample image data into the initial classification network one by one to obtain a MURA defect prediction probability, which is used to predict whether there is a MURA defect in the sample image data;

[0067] Calculate a classification loss value between the MURA defect prediction probability and the true label through the cross-entropy loss function, where the true label is obtained from the sample image data, and the cross-entropy loss function is as follows:

[0068] ；

[0069] where y i c is the classification label of the i-th image of the label classification sample, and p i c is the MURA defect probability value predicted by the classification network for the i-th image in the label classification sample, and N represents the number of images in the label classification sample;

[0070] Backpropagate the classification loss value to optimize the initial classification network, and input the next sample image data into the initial classification network until all the images included in the sample image data are input and the training is completed to obtain a classification network.

[0071] Optionally, the weighted fusion unit is specifically configured to:

[0072] Extract all feature layers of the initial CAM map;

[0073] Determine a high-level feature map and a low-level feature map from all the feature layers;

[0074] Perform weighted fusion on the high-level feature map and the low-level feature map through the following formula to obtain a binary refined CAM map:

[0075] (x)；

[0076] Among them, L is the total number of all the feature layers, (x) is the activation value of the c-th channel when the number of layers of the feature layer is 1, is the weight of the c-th channel when the number of layers of the feature layer is 1, is the weighting coefficient of the layer.

[0077] Optionally, the training unit is specifically further configured to:

[0078] Construct an initial segmentation network with the U-Net network as the backbone network, and define the loss function and the defective area weighting coefficient of the initial segmentation network;

[0079] Relabel the MURA defect segmentation area of the sample image data with inaccurate labels to obtain pseudo-label image data, where the labeled area of the pseudo-label image is the foreground area and the unlabeled area is the background area;

[0080] Train the initial segmentation network with the pseudo-label image data and the sample image data, and calculate the loss of the MURA defect segmentation area of the pseudo-label image data and the sample image data through a weakly supervised loss function to obtain a first segmentation loss value. The weakly supervised loss function is as follows:

[0081] ;

[0082] Among them, is the first segmentation loss value, is the defective area weighting coefficient, fg represents the foreground area, bg represents the background area, is the predicted value of the i-th image segmentation network of the pseudo-label image data and the sample image data;

[0083] Calculate the constrained loss function through the initial segmentation network to obtain a second segmentation loss value. The constrained loss function is as follows:

[0084] ；

[0085] Among them, is the second segmentation loss value, is the refined CAM map corresponding to the i-th image data of the pseudo-label image data and the sample image data, is the predicted value of the i-th image segmentation network of the pseudo-label image data and the sample image data.

[0086] Update the initial segmentation network with the first segmentation loss value and the second segmentation loss value until all the image data in the pseudo-label image data and the sample image data are completed for training, and obtain the segmentation network.

[0087] Optionally, the training unit is further specifically configured to:

[0088] Calculate a total loss value through the first segmentation loss value and the second segmentation loss value, and the total loss value is calculated by the following formula:

[0089] ；

[0090] Wherein, is the total loss value, , are the first segmentation loss value and the second segmentation loss value respectively weight coefficients;

[0091] Update the initial segmentation network according to the total loss value.

[0092] Optionally, the segmentation network includes a foreground network and a background network, and the defect area weighting coefficient of the foreground network is greater than that of the background network.

[0093] Optionally, the training unit is specifically configured to:

[0094] Obtain a first confidence level of the target CAM map through the foreground network;

[0095] Obtain a second confidence level of the target CAM map through the background network;

[0096] When the confidence levels of the first confidence level and the second confidence level are different, determine the confidence level state of the target CAM map as a low confidence level;

[0097] When the confidence levels of the first confidence level and the second confidence level are the same, determine the confidence level state of the target CAM map as a high confidence level.

[0098] A third aspect of the present application provides a MURA defect segmentation device, and the device includes:

[0099] A processor, a memory, an input / output unit, and a bus;

[0100] The processor is connected to the memory, the input / output unit, and the bus;

[0101] The memory stores a program, and the processor calls the program to execute the method of the first aspect and any optional method in the first aspect.

[0102] In the fourth aspect of the present application, a computer-readable storage medium is provided. A program is stored on the computer-readable storage medium, and when the program is executed on a computer, it executes the method in the first aspect and any optional method in the first aspect.

[0103] As can be seen from the above technical solutions, the present application has the following advantages:

[0104] The area containing MURA defects in the image to be segmented is roughly classified once through a classification network to obtain an initial CAM map of the image to be segmented. The feature layers of the initial CAM map are weighted and fused to obtain a binary fine CAM map that distinguishes the area of MURA defects from the normal area through binaryzation. After determining the binary fine CAM map, the binary fine CAM map is input into a segmentation network to segment the binary fine CAM map again. The obtained target CAM map contains the probability that each pixel block of the binary fine CAM map belongs to the MURA defect area. According to this probability, a low-confidence area is determined, and the low-confidence area is segmented. Each of the obtained refined areas is judged one by one through the MC-DropOut technique, and the refined areas that pass the judgment are determined as high-confidence areas. After all the refined areas are judged, the high-confidence areas are used as segmentation labels to segment the MURA defects in the image to be segmented. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] In order to more clearly illustrate the technical solutions in the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0106] Figure 1 It is a schematic flowchart of an embodiment of the MURA defect segmentation method in the present application;

[0107] Figure 2a It is a schematic flowchart of an embodiment of the first stage of the MURA defect segmentation method in the present application;

[0108] Figure 2b It is a schematic flowchart of an embodiment of the second stage of the MURA defect segmentation method in the present application;

[0109] Figure 3 It is a schematic flowchart of an embodiment of the training method of the classification network in the MURA defect segmentation method in the present application;

[0110] Figure 4 It is a schematic flowchart of an embodiment of the training method of the segmentation network in the MURA defect segmentation method in the present application;

[0111] Figure 5 It is a schematic structural diagram of an embodiment of the MURA defect segmentation system in this application;

[0112] Figure 6 It is a schematic structural diagram of an embodiment of the MURA defect segmentation device in this application. Specific implementation manners

[0113] It should be noted that a MURA defect segmentation method provided in this application can be applied to a terminal, a system, or a server. For example, the terminal can be a smart phone, a computer, a tablet computer, a smart TV, a smart watch, or a portable computer terminal, or it can also be a fixed terminal such as a desktop computer. For the convenience of description, in this application, the terminal is used as the execution subject for illustration.

[0114] Next, the technical solutions in this application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of this application. Obviously, the described embodiments are only a part of the embodiments of this application, rather than all the embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this application.

[0115] Please refer to Figure 1 , this application first provides an embodiment of the MURA defect segmentation method, and this embodiment includes:

[0116] S101. Obtain sample image data, where the sample image data is image data with pixel-level labeled MURA defect segmentation;

[0117] The sample image data is sample data for training a classification network and a segmentation network. The sample image data contains sufficient image data, and these image data contain pixel-level labeled MURA defect segmentation. The pixel-level labeled MURA defect segmentation is reflected as a segmentation label in the image data, that is, the edges of the MURA defects in the image data are reflected in the label state, and the MURA defect segmentation accuracy in the image data is pixel-level segmentation.

[0118] Because MURA defects usually appear as blurred and irregular regions, the MURA defect labels in the sample image data are generated through manual intervention to improve the accuracy of the MURA defect labels in the sample image data.

[0119] S102. Train a classification network and a segmentation network according to the sample image data. The classification network is used to classify whether an image contains MURA defects, and the segmentation network is used to segment the MURA defect region;

[0120] The classification network is used to classify whether there are MURA defects in the image data. In actual situations, the acquisition channels of the sample image data are the quality inspections of one or multiple batches of display production and the historical quality inspection records of the corresponding products on the relevant production lines. The sample image data contains a sufficient number of training samples to train the classification network and the segmentation network.

[0121] The model networks generated through training with the sample image data are the classification network and the segmentation network respectively. Among them, the classification network is used to distinguish whether the image contains MURA defects, and the segmentation network is used to segment the MURA defect area in the image, that is, to generate MURA defect labels.

[0122] S103. Obtain the image to be segmented, and input the image to be segmented into the classification network to obtain an initial CAM map;

[0123] After inputting the image to be segmented into the classification network, if the image to be segmented is an image containing MURA defects, the classification network classifies the image to be segmented by determining whether there are MURA defects in the image to be segmented. And in the process of obtaining the existence state of MURA defects, the classification network can determine the area where MURA defects may exist in the image to be segmented. At this time, this area will be marked as the label indicating that the image to be segmented belongs to the image containing MURA defects determined by the classification network. Therefore, when the image to be segmented is input into the classification network, an initial CAM map will be generated during its output. The initial CAM map is marked with the label of the area that may be a MURA defect area. This label is a rough mark, so the MURA defect in the image to be segmented cannot be segmented through this label.

[0124] S104. Extract all the feature layers of the initial CAM map, and perform weighted fusion on all the feature layers to obtain a binary fine CAM map;

[0125] After determining the initial CAM map, the terminal will refine the features in the CAM map according to the information of each feature layer of the initial CAM map.

[0126] Specifically, the feature map of each layer of the initial CAM map contains information at different levels. The low-level feature map captures details and edge information, while the high-level feature map contains more abstract semantic information. Through the weighted fusion of multiple feature maps, we can obtain a more detailed CAM map. Finally, a binary fine CAM map is generated. The binary fine CAM map will provide the probability of whether each pixel point is a defect area. These binary fine CAM maps binarize the probability of the defect area by designing a threshold and are used as pseudo-labels for subsequent segmentation tasks. That is, when the probability of this area being a MURA defect is greater than the designed threshold, the probability is recorded as 1, otherwise it is recorded as 0. By this recording method, the MURA defect area of the initial CAM map is binarized to obtain a binary fine CAM map.

[0127] The information carried in the binary refined CAM map includes all areas considered to have MURA defects. By refining the features, only the features related to MURA defects are included in the binary refined CAM map, and other features (such as the display screen edge) will be discarded to improve the acquisition efficiency of the features of the MURA defect area in the binary refined CAM map.

[0128] S105. Input the binary refined CAM map into the segmentation network to obtain the target CAM map;

[0129] According to the description of the foregoing steps, it can be seen that the MURA defect area in the binary refined CAM map is not perfect. Therefore, it is necessary to further refine the MURA defect area in the binary refined CAM map through the segmentation network.

[0130] Specifically, because the division of the MURA defect area in the foregoing steps is not perfect, the MURA defect area label of the binary refined CAM map obtained by the current terminal is defined as an inaccurate label. In this embodiment, the inaccurate label is regarded as a weakly supervised label through the weakly supervised loss, so as to judge the foreground area (i.e., the MURA defect area) and the background area (normal area) by changing the weights of the segmentation network. In this embodiment, since the foreground area is the defect area, giving a greater weight helps the network to better focus on the MURA defect, while the background area is given a smaller weight to avoid overfitting the background.

[0131] The image output by the segmentation network is the target CAM map. Compared with the binary refined CAM map, in the target CAM map, through multiple judgments of the foreground area and the background area, the probability of the MURA defect area will be calculated for the pixel points included in the label positions of all areas considered to be MURA defect areas. Therefore, the label of the MURA defect area in the target CAM map is non-binary, and the binary parameter therein will be replaced by the probability value.

[0132] S106. Determine the low-confidence areas in the target CAM map;

[0133] After obtaining the target CAM map, it is necessary to evaluate the confidence of each pixel point in the map belonging to the MURA defect area. The terminal will determine the area below the preset probability value through the probability value of the MURA defect area label. The process of determining the preset confidence area is to repeatedly obtain the target CAM map, that is, to obtain the probability value of the MURA area by modifying the weight distribution of the segmentation network multiple times for the picture to be segmented. When the average probability value within a MURA area is less than the preset probability value, it is determined that the area belongs to the low-confidence area.

[0134] S107. Segment the low-confidence areas to obtain the refined areas;

[0135] When the average probability value of a MURA defect area is less than the preset probability value, it indicates that it cannot be determined whether this area is a MURA defect area. At this time, the terminal needs to refine this area through the superpixel technology. The superpixel technology is to segment the image into multiple smaller areas, thereby improving the segmentation accuracy of the network in the detail areas.

[0136] In this embodiment, the terminal will segment the low-confidence area through the superpixel technology to obtain a refined area. This refined area is obtained by subdividing the low-confidence area, aiming to refine the area that may be a MURA defect.

[0137] S108: Randomly copy the refined area to the normal area, and generate multiple prediction results through the MC-DropOut technology;

[0138] Randomly copy the refined area to the normal area in the image to test the prediction consistency of the model for these areas. Apply the Monte Carlo DropOut (MC-DropOut) technology to generate multiple different prediction results by randomly discarding neurons during the inference process.

[0139] S109: Analyze the consistency of the multiple prediction results, and output the MURA defect status of the refined area;

[0140] Analyze the multiple prediction results generated by MC-DropOut, and check whether the refined area is consistently marked as a MURA defect area in all predictions. If the refined area is marked as a MURA defect area in all predictions, output the MURA defect status of this area as having a MURA defect, and execute step S110; otherwise, output as not having a MURA defect.

[0141] S110: When the MURA defect status is having a MURA defect, change the refined area to a high-confidence area;

[0142] If the refined area is determined to have a MURA defect, update it from the low-confidence area to the high-confidence area. Update the target CAM map, and update the label of the refined area from low-confidence to high-confidence to reflect a more accurate MURA defect segmentation result.

[0143] S111: Use the high-confidence area as the segmentation label to perform MURA defect segmentation on the image to be segmented.

[0144] If the refined area is determined to have a MURA defect, update it from the low-confidence area to the high-confidence area. Update the target CAM map, and update the label of the refined area from low-confidence to high-confidence to reflect a more accurate MURA defect segmentation result.

[0145] The area containing MURA defects in the image to be segmented is roughly classified once through a classification network to obtain the initial CAM map of the image to be segmented. The feature layers of the initial CAM map are weighted and fused to obtain a binary fine CAM map that distinguishes the area of MURA defects from the normal area through binaryzation. After determining the binary fine CAM map, the binary fine CAM map is input into the segmentation network to segment the binary fine CAM map again, and the obtained target CAM map contains the probability that each pixel block of the binary fine CAM map belongs to the MURA defect area. According to this probability, the low-confidence area is determined and segmented, and each of the obtained refined areas is judged one by one through the MC-DropOut technique. The refined areas that pass the judgment are determined as high-confidence areas. After all the refined areas are judged, the high-confidence areas are used as segmentation labels to segment the MURA defects in the image to be segmented.

[0146] Please refer to Figure 2a and Figure 2b , another embodiment of the MURA defect segmentation method is provided in the embodiment of the present application. This embodiment includes:

[0147] S201. Obtain sample image data, where the sample image data is image data with pixel-level annotation for MURA defect segmentation;

[0148] S202. Train a classification network and a segmentation network according to the sample image data. The classification network is used to classify whether an image contains MURA defects, and the segmentation network is used to segment the MURA defect area;

[0149] S203. Obtain the image to be segmented, and input the image to be segmented into the classification network to obtain the initial CAM map;

[0150] Steps S201 to S203 in this embodiment are similar to steps S101 to S103 in the foregoing embodiment, and will not be elaborated here specifically.

[0151] S204. Extract all feature layers of the initial CAM map;

[0152] A feature map is the output of a convolutional layer in a convolutional neural network (CNN). It represents a set of mappings obtained after the input image is processed by the convolutional layer. Each feature map captures different features or patterns of the input image, such as edges, textures, shapes, etc. Each element of the feature map corresponds to the feature response of a local area in the input image. The characteristics of the feature map include:

[0153] Multi-channel: A convolutional layer usually contains multiple filters (also known as convolutional kernels), and each filter generates a feature map. Therefore, the output of a convolutional layer is a set of multi-channel feature maps.

[0154] Dimensionality reduction: After being processed by a pooling layer (such as max pooling or average pooling), the spatial size of the feature map usually decreases, thereby reducing the number of parameters and computational complexity.

[0155] Hierarchical: In a deep CNN, as the network depth increases, the feature maps gradually transition from capturing low-level features (such as edges and textures) to capturing higher-level features (such as object parts).

[0156] The generation of the CAM map depends on the feature maps output by the convolutional layer. The feature maps provide local feature information of the image, while the CAM map is a weighted sum of this feature information, used to highlight the regions that are most important for the classification decision.

[0157] Specifically, the terminal extracts the feature maps of all layers from the initial CAM map generated by the classification network. These feature maps contain information at different levels from low to high, and each layer provides information about different aspects of the image.

[0158] S205. Determine the high-level feature maps and low-level feature maps from all feature layers;

[0159] According to the position of the feature layer in the network, the feature maps are divided into high-level feature maps and low-level feature maps. Low-level feature maps usually contain more details and edge information about the image, while high-level feature maps contain more abstract semantic information.

[0160] S206. Perform weighted fusion on the high-level feature maps and low-level feature maps through the following formula to obtain a binary refined CAM map:

[0161] (x) ;

[0162] where L is the total number of all feature layers, (x) is the activation value of the c-th channel when the layer number of the feature layer is 1, is the weight of the c-th channel when the layer number of the feature layer is 1, is the weighted coefficient of the layer.

[0163] The binary refined CAM (BR-CAM) is a processed class activation map, which is used to more accurately locate and segment the MURA defect regions in the image.

[0164] In the initial CAM map before weighted fusion, the marking of MURA defects directly covers and marks areas that may be MURA defects. However, the scope of MURA defects themselves is difficult to determine. Therefore, this marking, when distinguished by fineness, will be classified as a rough marking. Binarize the information in the CAM map.

[0165] S207. Input the binarized fine CAM map into the segmentation network to obtain the target CAM map.

[0166] The segmentation network includes a foreground network and a background network. The weighted coefficient of the defect area in the foreground network is greater than that in the background network.

[0167] Input the binarized fine CAM map into the segmentation network (which includes a foreground network and a background network) to further refine the MURA defect area. The segmentation network obtains multiple results by modifying the weights of the foreground (defect area) and the background (normal area) to improve the accuracy of defect segmentation. Among them, the weighted coefficient of the defect area in the foreground network is greater than that in the background network.

[0168] S208. Obtain the first confidence level of the target CAM map through the foreground network.

[0169] Use the foreground network in the segmentation network to evaluate the confidence level of each pixel in the target CAM map belonging to the MURA defect area to obtain the first confidence level.

[0170] S209. Obtain the second confidence level of the target CAM map through the background network.

[0171] Use the background network in the segmentation network to evaluate the confidence level of each pixel in the target CAM map belonging to the MURA defect area to obtain the second confidence level.

[0172] It should be noted that since the weights of MURA defects change in the foreground network and the background network, which in turn affects the fitting results of the segmentation network model, the confidence levels of the target CAM maps output by different networks for MURA defects will be different. Distinguished by the foreground network and the background network, the obtained confidence levels are the first confidence level and the second confidence level respectively.

[0173] S210. When the confidence levels of the first confidence level and the second confidence level are different, determine the confidence level state of the target CAM map as a low confidence level.

[0174] In actual situations, there is a pre-set confidence level value in the terminal memory. That is, taking this pre-set confidence level as the boundary, if the first confidence level and / or the second confidence level is higher than the pre-set confidence level, then determine the first confidence level and / or the second confidence level as a high confidence level, otherwise determine it as a low confidence level.

[0175] In actual situations, the terminal determines the states corresponding to the confidence levels based on the coverage status of the coarse labels, namely the foreground confidence level and the background confidence level. The foreground confidence level is the area covered by the coarse label, that is, the MURA defect area. Therefore, the first confidence level and the second confidence level obtained correspond to the confidence level that this area is the MURA defect area; the background confidence level is the area not covered by the coarse label, that is, the normal area. Therefore, the first confidence level and the second confidence level obtained correspond to the confidence level that this area is the normal area.

[0176] S211. When the confidence levels of the first confidence level and the second confidence level are the same, determine the confidence level state of the target CAM map as a high confidence level.

[0177] Similar to step S210, if the evaluation results of the foreground network and the background network for the same pixel are consistent, that is, both consider that this pixel belongs to the defect area or both consider it belongs to the normal area, then mark the confidence level state of this pixel as a high confidence level.

[0178] S212. Segment the low-confidence area to obtain a refined area;

[0179] S213. Randomly copy the refined area to the normal area and generate multiple prediction results through the MC-DropOut technique;

[0180] S214. Analyze the consistency of the multiple prediction results and output the MURA defect state of the refined area;

[0181] S215. When the MURA defect state is that there is a MURA defect, change the refined area to a high-confidence area;

[0182] S216. Use the high-confidence area as a segmentation label to perform MURA defect segmentation on the image to be segmented.

[0183] Steps S212 to S216 in this embodiment are similar to steps S107 to S111 in the foregoing embodiment, and will not be elaborated here specifically.

[0184] In this embodiment, the terminal analyzes the same target CAM map multiple times through multiple segmentation networks with different weights to improve the accuracy of segmenting the target CAM map.

[0185] Specifically, in step S102, it is necessary to train the classification network and the segmentation network. For the specific training process of the classification network, please refer to Figure 3 , Figure 3 This is an embodiment of the training method of the classification network in the MURA defect segmentation method provided by the embodiments of the present application. This embodiment includes:

[0186] S301. Initialize an initial classification network according to preset training parameters, and define the loss function of the initial classification network as a cross-entropy loss function;

[0187] First, the terminal initializes an initial classification network according to preset training parameters (such as learning rate, batch size, network architecture, etc.). The terminal defines the loss function of the initial classification network as a cross-entropy loss function. The cross-entropy loss function is a loss function for classification tasks, which measures the difference between the predicted probability distribution of the model and the true label distribution.

[0188] S302. Input sample image data into the initial classification network one by one to obtain the MURA defect prediction probability, which is used to predict whether there is a MURA defect in the sample image data;

[0189] Input the sample image data into the initial classification network one by one. These image data have been pixel-level annotated and are used to train the network to identify MURA defects. The network processes each input image and outputs the prediction probability of MURA defects. This probability value represents the possibility of the existence of MURA defects in the image.

[0190] S303. Calculate the classification loss value between the MURA defect prediction probability and the true label through the cross-entropy loss function. The true label is obtained from the sample image data, and the cross-entropy loss function is as follows:

[0191] ;

[0192] where, y i c is the classification label of the i-th image of the label classification sample, and p i c is the MURA defect probability value predicted by the i-th image in the label classification sample through the classification network. N represents the number of images in the label classification sample;

[0193] Use the cross-entropy loss function to calculate the difference between the predicted probability and the true label. This loss value reflects the prediction error of the model in the current state.

[0194] S304. Optimize the initial classification network by backpropagating the classification loss value, and input the next sample image data into the initial classification network until all the images included in the sample image data are input and the training is completed to obtain the classification network.

[0195] During backpropagation, according to the calculated classification loss value, perform backpropagation through the network to calculate the gradient of each parameter. The purpose of parameter optimization is to update the network parameters according to the gradient through the optimization algorithm to reduce the loss value and improve the prediction accuracy of the model.

[0196] Repeat the above process until all sample image data are input into the network for training. This process may require multiple iterations until the model performance meets the requirements or reaches the preset stopping conditions.

[0197] This embodiment is used to train a classification network. The classification network extracts features layer by layer from the feature layer to determine whether the image contains a MURA defect area and classifies the image according to the presence status of the MURA defect area.

[0198] Specifically, in step S102, it is necessary to train the classification network and the segmentation network. For the specific training process of the segmentation network, please refer to Figure 4 , Figure 4 This application embodiment provides an embodiment of the training method of the segmentation network in the MURA defect segmentation method. This embodiment includes:

[0199] S401. Construct an initial segmentation network with the U-Net network as the backbone network, and define the loss function and the defect area weighting coefficient of the initial segmentation network;

[0200] U-Net is a network architecture for image segmentation, which is particularly suitable for image segmentation tasks. Define the loss function and the defect area weighting coefficient of the initial segmentation network. The loss function is used to measure the difference between the prediction result and the true label, and the weighting coefficient is used to adjust the importance of different regions (such as the foreground region and the background region) in the loss calculation.

[0201] S402. Relabel the MURA defect segmentation area of the sample image data with inaccurate labels to obtain pseudo-label image data. The labeled areas of the pseudo-label image are foreground regions, and the unlabeled areas are background regions;

[0202] Relabel the MURA defect segmentation area of the sample image data with inaccurate labels to generate pseudo-label image data. In these pseudo-label images, the labeled areas are foreground regions (defect areas), and the unlabeled areas are background regions.

[0203] In actual situations, the labels of the MURA defect segmentation areas carried in the sample image data are accurate strong labels. To improve the segmentation ability of the segmentation network for inaccurate weak labels, during training, it will be trained with weak label data similar to those labeled by the classification network, and the sample image data will be used as the target for segmentation to train the initial segmentation network.

[0204] S403. Train the initial segmentation network with the pseudo-label image data and the sample image data, and calculate the loss of the MURA defect segmentation area of the pseudo-label image data and the sample image data through the weakly supervised loss function to obtain the first segmentation loss value. The weakly supervised loss function is as follows:

[0205] ;

[0206] Among them, is the first segmentation loss value, is the defect area weighting coefficient, fg represents the foreground area, and bg represents the background area, is the prediction value of the i-th image of the segmentation network for the pseudo-label image data and the sample image data;

[0207] Update the initial segmentation network through the first segmentation loss value and the second segmentation loss value until all the image data in the pseudo-label image data and the sample image data are completed training to obtain the segmentation network.

[0208] The weakly supervised loss function is used to calculate the difference between the prediction value and the pseudo-label, where the different weights of the foreground and background areas are considered.

[0209] Train the initial segmentation network using the pseudo-label image data and the sample image data. Calculate the loss value of the MURA defect segmentation area through the weakly supervised loss function, which considers the defect area weighting coefficient to better focus on the defect area.

[0210] S404. Calculate through the initial segmentation network using the constraint loss function to obtain the second segmentation loss value. The constraint loss function is as follows:

[0211] ;

[0212] Among them, is the second segmentation loss value, is the refined CAM map corresponding to the i-th image data of the pseudo-label image data and the sample image data, is the prediction value of the i-th image of the segmentation network for the pseudo-label image data and the sample image data;

[0213] Calculate through the initial segmentation network using the constraint loss function to obtain the second segmentation loss value. This loss function is used to measure the consistency between the prediction result of the segmentation network and the refined CAM map to further guide the learning of the segmentation network.

[0214] S405. Calculate the total loss value through the first segmentation loss value and the second segmentation loss value. The total loss value is calculated by the following formula:

[0215] ;

[0216] Among them, is the total loss value, , are the first segmentation loss value and the second segmentation loss value respectively Weight coefficient;

[0217] Calculate the total loss value through the first segmentation loss value and the second segmentation loss value. The total loss value is the weighted sum of these two loss values, where the weight coefficient is used to balance the influence of these two loss values.

[0218] S406. Update the initial segmentation network according to the total loss value to obtain the segmentation network.

[0219] Update the initial segmentation network according to the total loss value, so that the terminal adjusts the parameters of the segmentation network according to the total loss value to minimize the total loss value, thereby improving the performance of the segmentation network.

[0220] By repeating the update process of steps S403 to S406 until all image data is completed for training, an optimized segmentation network is finally obtained.

[0221] This embodiment is a specific training process of a segmentation network, and this segmentation network can perform more accurate MURA defect region segmentation on image data containing only weak labels of MURA defects.

[0222] The MURA defect segmentation method in the embodiments of the present application has been described in detail above. Next, the MURA defect segmentation system and device will be described in detail.

[0223] Please refer to Figure 5 , an embodiment of the MURA defect segmentation system is provided in the embodiments of the present application, and this embodiment includes:

[0224] The first acquisition unit 501 is used to acquire sample image data, and the sample image data is image data that has completed pixel-level annotation of MURA defect segmentation;

[0225] The training unit 502 is used to train a classification network and a segmentation network according to the sample image data. The classification network is used to classify whether an image contains MURA defects, and the segmentation network is used to segment the MURA defect region;

[0226] The second acquisition unit 503 is used to acquire the image to be segmented, and input the image to be segmented into the classification network to obtain the initial CAM map;

[0227] The weighted fusion unit 504 is used to extract all feature layers of the initial CAM map, and perform weighted fusion on all feature layers to obtain a binary fine CAM map;

[0228] The input unit 505 is used to input the binary fine CAM map into the segmentation network to obtain the target CAM map;

[0229] The determination unit 506 is used to determine the low-confidence region in the target CAM map;

[0230] The first segmentation unit 507 is used to segment the low-confidence region to obtain a refined region;

[0231] The prediction unit 508 is used to randomly copy the refined region to the normal region and generate multiple prediction results through the MC-DropOut technique;

[0232] The analysis unit 509 is used to analyze the consistency of multiple prediction results and output the MURA defect status of the refined region;

[0233] The change unit 510 is used to change the refined region to a high-confidence region when the MURA defect status is that there is a MURA defect;

[0234] The second segmentation unit 511 is used to perform MURA defect segmentation on the image to be segmented with the high-confidence region as the segmentation label.

[0235] In this embodiment, the training unit 502 is specifically used for:

[0236] Initialize the initial classification network according to the preset training parameters, and define the loss function of the initial classification network as the cross-entropy loss function;

[0237] Input the sample image data into the initial classification network one by one to obtain the MURA defect prediction probability, and the MURA defect prediction probability is used to predict whether there is a MURA defect in the sample image data;

[0238] Calculate the classification loss value between the MURA defect prediction probability and the true label through the cross-entropy loss function. The true label is obtained from the sample image data, and the cross-entropy loss function is as follows:

[0239] ;

[0240] Among them, y i c is the classification label of the i-th image of the label classification sample, and p i c is the MURA defect probability value predicted by the i-th image in the label classification sample through the classification network, and N represents the number of images in the label classification sample;

[0241] Optimize the initial classification network by backpropagating the classification loss value, and input the next sample image data into the initial classification network until all the images included in all the sample image data are input and the training is completed to obtain the classification network.

[0242] In this embodiment, the weighted fusion unit 504 is specifically used for:

[0243] Extract all the feature layers of the initial CAM map;

[0244] Determine the high-level feature map and the low-level feature map from all the feature layers;

[0245] Perform weighted fusion on the high-level feature map and the low-level feature map through the following formula to obtain a binarized fine CAM map:

[0246] (x) ;

[0247] where L is the total number of all the feature layers, (x) is the activation value of the c-th channel when the number of layers of the feature layer is 1, is the weight of the c-th channel when the number of layers of the feature layer is 1, is the weighted coefficient of the layer.

[0248] In this embodiment, the training unit 502 is specifically further configured to:

[0249] Construct an initial segmentation network with the U-Net network as the backbone network, and define the loss function and the defect area weighted coefficient of the initial segmentation network;

[0250] Relabel the MURA defect segmentation area of the sample image data through inaccurate labels to obtain pseudo-label image data, where the labeled area of the pseudo-label image is the foreground area and the unlabeled area is the background area;

[0251] Train the initial segmentation network through the pseudo-label image data and the sample image data, and calculate the loss of the MURA defect segmentation area of the pseudo-label image data and the sample image data through the weakly supervised loss function to obtain a first segmentation loss value. The weakly supervised loss function is as follows:

[0252] ;

[0253] where, is the first segmentation loss value, is the defect area weighted coefficient, fg represents the foreground area, bg represents the background area, is the predicted value of the i-th image segmentation network of the pseudo-label image data and the sample image data;

[0254] Calculate the constrained loss function through the initial segmentation network to obtain A second segmentation loss value. The constrained loss function is as follows:

[0255] ;

[0256] where, is the second segmentation loss value, is the fine CAM map corresponding to the i-th image data of the pseudo-label image data and the sample image data. is the predicted value of the i-th image segmentation network of the pseudo-label image data and the sample image data.

[0257] Update the initial segmentation network through the first segmentation loss value and the second segmentation loss value until all the image data in the pseudo-label image data and the sample image data are completed for training, and obtain the segmentation network.

[0258] In this embodiment, the training unit 502 is further specifically configured to:

[0259] Calculate the total loss value through the first segmentation loss value and the second segmentation loss value. The total loss value is calculated by the following formula:

[0260] ;

[0261] Wherein, is the total loss value, , are the first segmentation loss value and the second segmentation loss value respectively weight coefficients;

[0262] Update the initial segmentation network according to the total loss value.

[0263] In this embodiment, the segmentation network includes a foreground network and a background network, and the defect area weighting coefficient of the foreground network is greater than that of the background network.

[0264] In this embodiment, the training unit 502 is specifically configured to:

[0265] Obtain the first confidence level of the target CAM map through the foreground network;

[0266] Obtain the second confidence level of the target CAM map through the background network;

[0267] When the confidence levels of the first confidence level and the second confidence level are different, determine the confidence level state of the target CAM map as a low confidence level;

[0268] When the confidence levels of the first confidence level and the second confidence level are the same, determine the confidence level state of the target CAM map as a high confidence level.

[0269] In this embodiment, the functions of each unit correspond to the steps in the foregoing Figures 1 to 4 illustrated embodiment, and will not be elaborated here.

[0270] Please refer to Figure 6 , another embodiment of the MURA defect segmentation device provided by the embodiment of the present application includes:

[0271] A processor 601, a memory 602, an input / output unit 603, and a bus 604;

[0272] The processor 601 is connected to the memory 602, the input / output unit 603, and the bus 604;

[0273] The processor 601 specifically executes Figures 1 to 4 the operations corresponding to the steps in the method, which will not be elaborated here specifically.

[0274] This application also relates to a computer-readable storage medium, on which a program is stored. When the program runs on a computer, it causes the computer to execute any of the above methods.

[0275] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated here.

[0276] In several embodiments provided by this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces, and the indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0277] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or they can be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0278] In addition, in each embodiment of this application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated units can be implemented in the form of hardware or in the form of software functional units.

[0279] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories (ROM, read-only memory), random access memories (RAM, random access memory), magnetic disks, or optical discs.

Claims

1. A MURA defect segmentation method, characterized in that: The method comprises: Acquire sample image data, where the sample image data is image data that has completed pixel-level annotation MURA defect segmentation; Training a classification network and a segmentation network according to the sample image data, wherein the classification network is used to classify whether an image contains a MURA defect, and the segmentation network is used to segment the MURA defect area; Acquire an image to be segmented, and input the image to be segmented into the classification network to obtain an initial CAM image; Extracting all feature layers of the initial CAM image, and performing weighted fusion on all feature layers to obtain a binary refined CAM image; Inputting the binarized fine CAM image into the segmentation network to obtain a target CAM image; Determining a low confidence region in the target CAM map; Segmenting the low confidence region to obtain a refined region; The refined area is randomly copied to the normal area, and multiple prediction results are generated by MC-DropOut technology; Analyzing the consistency of the multiple prediction results and outputting the MURA defect status of the refined area; When the MURA defect state is that a MURA defect exists, changing the refined area to a high confidence area; Performing MURA defect segmentation on the image to be segmented using the high confidence region as a segmentation label; Training a segmentation network according to the sample image data includes: An initial segmentation network is constructed using a U-Net network as a backbone network, and a loss function and a defect area weighting coefficient of the initial segmentation network are defined; Re-labeling the MURA defect segmentation area of ​​the sample image data by using an imprecise label to obtain pseudo-label image data, wherein the labeled portion of the pseudo-label image is a foreground area and the unlabeled portion is a background area; The initial segmentation network is trained by the pseudo-label image data and the sample image data, and the loss of the MURA defect segmentation area is calculated for the pseudo-label image data and the sample image data by a weakly supervised loss function to obtain a first segmentation loss value, wherein the weakly supervised loss function is as follows: ； in, is the first segmentation loss value, is the weighting coefficient of the defect area, fg represents the foreground area, bg represents the background area, is the predicted value of the i-th image segmentation network of the pseudo-label image data and the sample image data, is the pseudo-label actual value of the i-th image of the pseudo-label image data and the sample image data; The constraint loss function is calculated by the initial segmentation network, and we get The second segmentation loss value, the constraint loss function is as follows: ； in, is the second segmentation loss value, is a fine CAM image corresponding to the pseudo-label image data and the i-th image data of the sample image data, is the predicted value of the i-th image segmentation network of the pseudo-label image data and the sample image data; The initial segmentation network is updated by using the first segmentation loss value and the second segmentation loss value until all image data in the pseudo-label image data and the sample image data are trained to obtain a segmentation network.

2. The MURA defect segmentation method according to claim 1, characterized in that: The step of training a classification network according to the sample image data comprises: Initialize an initial classification network according to preset training parameters, and define a loss function of the initial classification network as a cross entropy loss function; Inputting the sample image data into the initial classification network one by one to obtain a MURA defect prediction probability, wherein the MURA defect prediction probability is used to predict whether the sample image data has a MURA defect; The classification loss value between the MURA defect prediction probability and the true label is calculated by the cross entropy loss function, and the true label is obtained by the sample image data. The cross entropy loss function is as follows: ； Among them, y i c is the classification label of the i-th image of the label classification sample, p i c is the MURA defect probability value predicted by the classification network for the i-th image in the label classification sample, and N represents the number of images in the label classification sample; The classification loss value is back-propagated to optimize the initial classification network, and the next sample image data is input into the initial classification network until all image inputs included in the sample image data are trained to obtain a classification network.

3. The MURA defect segmentation method according to claim 1, characterized in that: The step of extracting all feature layers of the initial CAM image and performing weighted fusion on all feature layers to obtain a binary refined CAM image includes: Extracting all feature layers of the initial CAM image; Determine a high-level feature map and a low-level feature map from all the feature layers; The high-level feature map and the low-level feature map are weightedly fused by the following formula to obtain a binary fine CAM map: (x) ； Where L is the total number of all feature layers, (x) is the activation value of the cth channel when the number of feature layers is 1, is the number of feature layers, 1, and the weight of the cth channel. is the weight coefficient of the layer, and C represents the total number of channels of each feature layer.

4. The MURA defect segmentation method according to claim 1, characterized in that: The updating of the initial segmentation network by using the first segmentation loss value and the second segmentation loss value comprises: The total loss value is calculated by the first segmentation loss value and the second segmentation loss value, and the total loss value is calculated by the following formula: ； in, is the total loss value, , are the first segmentation loss value and the second segmentation loss value respectively The weight coefficient of The initial segmentation network is updated according to the total loss value.

5. The MURA defect segmentation method according to any one of claims 1 to 4, characterized in that: The segmentation network includes a foreground network and a background network, and a weighted coefficient of a defect area of ​​the foreground network is greater than a weighted coefficient of a defect area of ​​the background network.

6. The MURA defect segmentation method according to claim 5, characterized in that: Determining the low confidence area in the target CAM map includes: Acquire a first confidence of the target CAM image through a foreground network; Acquire a second confidence of the target CAM image through the background network; When the confidence states of the first confidence and the second confidence are different, determining the confidence state of the target CAM map as low confidence; When the confidence states of the first confidence and the second confidence are the same, the confidence state of the target CAM map is determined to be high confidence.

7. A MURA defect segmentation system, characterized in that: The MURA defect segmentation system comprises: A first acquisition unit is used to acquire sample image data, where the sample image data is image data that has completed pixel-level annotation MURA defect segmentation; A training unit, used for training a classification network and a segmentation network according to the sample image data, wherein the classification network is used for classifying whether an image contains a MURA defect, and the segmentation network is used for segmenting the MURA defect area; A second acquisition unit is used to acquire the image to be segmented, and input the image to be segmented into the classification network to obtain an initial CAM image; A weighted fusion unit, used for extracting all feature layers of the initial CAM image, and performing weighted fusion on all feature layers to obtain a binary refined CAM image; An input unit, used for inputting the binarized fine CAM image into the segmentation network to obtain a target CAM image; A determination unit, used to determine a low confidence area in the target CAM map; A first segmentation unit, used to segment the low confidence region to obtain a refined region; A prediction unit, used for randomly copying the refined area to a normal area and generating multiple prediction results by using the MC-DropOut technology; An analysis unit, configured to analyze the consistency of the plurality of prediction results and output the MURA defect state of the refined area; a changing unit, configured to change the refined region into a high confidence region when the MURA defect state indicates that a MURA defect exists; A second segmentation unit is used to perform MURA defect segmentation on the image to be segmented by using the high confidence region as a segmentation label; The training unit is further specifically used for: An initial segmentation network is constructed using a U-Net network as a backbone network, and a loss function and a defect area weighting coefficient of the initial segmentation network are defined; Re-labeling the MURA defect segmentation area of ​​the sample image data by using an imprecise label to obtain pseudo-label image data, wherein the labeled portion of the pseudo-label image is a foreground area and the unlabeled portion is a background area; The initial segmentation network is trained by the pseudo-label image data and the sample image data, and the loss of the MURA defect segmentation area is calculated for the pseudo-label image data and the sample image data by a weakly supervised loss function to obtain a first segmentation loss value, wherein the weakly supervised loss function is as follows: ； in, is the first segmentation loss value, is the weighting coefficient of the defect area, fg represents the foreground area, bg represents the background area, is the predicted value of the i-th image segmentation network of the pseudo-label image data and the sample image data, is the pseudo-label actual value of the i-th image of the pseudo-label image data and the sample image data; The constraint loss function is calculated by the initial segmentation network, and we get The second segmentation loss value, the constraint loss function is as follows: ； in, is the second segmentation loss value, is a fine CAM image corresponding to the pseudo-label image data and the i-th image data of the sample image data, is the predicted value of the i-th image segmentation network of the pseudo-label image data and the sample image data; The initial segmentation network is updated by using the first segmentation loss value and the second segmentation loss value until all the image data in the pseudo-label image data and the sample image data are trained to obtain a segmentation network; The MURA defect segmentation system performs the method as described in any one of claims 1 to 5 when executed.

8. A MURA defect segmentation device, characterized in that: The device comprises: Processor, memory, input-output unit, and bus; The processor is connected to the memory, the input and output unit, and the bus; The memory stores a program, and the processor calls the program to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, and when the program is executed on a computer, the method according to any one of claims 1 to 6 is performed.

Images