Metal plate ray image noise label detection method, system, equipment and medium

Abstract

The invention discloses a metal plate ray image noise label detection method, system and device and a medium. The method comprises the following steps: marking whether each label in a training data set is a clean label or a noise label by using an industrial defect visual language model; putting the picture label pairs of which the labels are marked as clean labels into a clean subset which is initialized to be empty, and pre-training two CNN models by utilizing the clean subset to obtain two CNN models with preliminary discrimination capability; formally training two CNN models with preliminary discrimination capability by using the training data set, selecting picture label pairs with clean labels from the training data set in each formal training, and putting the picture label pairs into clean subsets until the maximum training round is reached; solving an intersection of v clean subsets obtained by the last v training rounds of formal training to obtain a final clean sample set; and training a reinitialized CNN model by using the clean sample set, wherein the trained CNN model is used for metal plate defect classification. According to the invention, the cost of manual label labeling is reduced.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and computer vision, and specifically relates to a method, system, device and medium for detecting noise labels in metal plate radiographic images. Background Art

[0002] Convolutional Neural Networks (CNN) are widely used in metal plate defect recognition tasks. Existing defect recognition methods mainly rely on X-ray nondestructive testing technology to generate scanned radiographic images, and then use CNN to identify and classify defects in radiographic images. However, CNN training requires a large number of radiographic images with accurate labels. During processing, metal plates are usually blurred in some areas due to surface smoothness and processing environment interference, and cannot be accurately distinguished as abnormal or normal. In addition, they are separated from the large-scale manual labeling process, resulting in a large number of incorrectly labeled training data in the dataset (noise labels). Existing studies have shown that a large number of training samples with noisy labels can easily cause a sharp drop in the discrimination performance and generalization performance of CNN, and large-scale manual labeling is costly, so there is an urgent need for a class of automated noise label detection methods. Summary of the invention

[0003] In view of the deficiencies in the prior art, the present invention provides a method, system, device and medium for detecting noise labels in metal plate X-ray images, introduces AnomalyGPT, the latest large-scale visual language model in the field of industrial anomaly detection, and uses the large model to assist in noise label screening, thereby saving manual labeling costs.

[0004] To achieve the above object, the present invention adopts the following technical solutions:

[0005] A method for detecting noise labels in a metal plate radiographic image comprises the following steps:

[0006] S1. Using the industrial defect visual language model to mark each label in the training data set as a clean label or a noise label; the training data set includes a plurality of image-label pairs, each image-label pair includes a metal plate X-ray scan image and a label, and the label is used to mark the defect type of the metal plate X-ray scan image;

[0007] S2, putting the image label pairs marked as clean labels into the clean subset that has been initialized to empty, and using the clean subset to pre-train two CNN models to obtain two CNN models with preliminary discrimination capabilities;

[0008] S3. Use the training data set to formally train two CNN models with preliminary discrimination capabilities. In each round of formal training, image label pairs with clean labels are selected from the training data set and put into the clean subset. A clean subset is obtained in each round of formal training until the maximum number of training rounds is reached.

[0009] S4, finding the intersection of the v clean subsets obtained after the last v training rounds of formal training to obtain the final clean sample set;

[0010] S5. Use the clean sample set to train a reinitialized CNN model, and the trained CNN model is used for metal plate defect classification.

[0011] To optimize the above technical solutions, the specific measures taken also include:

[0012] Further, in S1, the use of the industrial defect visual language model to mark each label in the training data set as a clean label or a noise label is specifically:

[0013] Set a prompt word for the industrial defect visual language model, and set the prompt word to "This is a X-ray scan image of a metal plate. The metal plate contains {the given label of the image Corresponding defect type name}? Please answer yes or no. ";

[0014] The training data set Input the industrial defect visual language model and take the label in the image label pair with the answer "yes" as the clean label, where is the i-th metal plate X-ray scanning image, is the i-th label, and n is the number of image-label pairs.

[0015] Furthermore, S3 specifically includes:

[0016] S3.1. At the beginning of the current training round, two CNN models with preliminary discrimination capabilities are used to calculate the training data set The cross entropy loss of each pair of image labels in , and the loss sequence is obtained ;

[0017]

[0018] In the formula, is the i-th metal plate X-ray scanning image, is the i-th label, n is the number of image label pairs, It means that the cross entropy loss is calculated for the i-th image label pair using the first CNN model with preliminary discriminative ability. Indicates that the cross entropy loss is calculated for the i-th image label pair using the second CNN model with preliminary discriminative ability; represents the cross entropy loss of the i-th image label pair;

[0019] S3.2. Use a two-component Gaussian mixture model to model the loss sequence Fitting is performed to estimate the probability that the label in all image label pairs is a clean label, and a probability sequence is obtained , where Represents the cross entropy loss of the Gaussian mixture model corresponding to the component g with the smallest mean for the i-th pair of image labels after fitting the distribution of the loss sequence Q The predicted probability of is the probability that the label in the i-th image label pair is a clean label;

[0020] S3.3. Copy the clean subset generated by S2 , A clean subset labeled for industrial defect visual language model, is the clean subset of the current training round t, and based on the probability sequence W, selects the image label pairs whose probability of being a clean label is greater than the threshold and puts them into , get the clean subset of the current training round t ;

[0021] S3.4. Two CNN models with preliminary discrimination capabilities turn on the dropout function, randomly discard some neurons, and use the clean subset Supervised training of two CNN models with preliminary discriminative capabilities. During training, the AutoAugment strategy is used to enhance the data of input image label pairs, and the training loss is the cross entropy loss.

[0022] S3.5. Turn off the drop out operation and increment the training round t by 1. If t does not reach the maximum training round , then jump to S3.1, otherwise execute S4.

[0023] Furthermore, in S4, the intersection of the v clean subsets obtained in the last v training rounds of the formal training is obtained to obtain the final clean sample set, which is expressed as follows:

[0024]

[0025] In the formula, is the final clean sample set, is the i-th image label pair, is the i-th metal plate X-ray scanning image, is the i-th label, is the intersection of v clean subsets obtained after v training rounds, For the The clean subset obtained after training rounds; is the maximum number of training times, is the training data set.

[0026] The present invention also provides a metal plate radiographic image noise label detection system, comprising:

[0027] An industrial defect visual language model is used to mark whether each label in a training data set is a clean label or a noise label; the training data set includes a plurality of image-label pairs, each image-label pair includes a metal plate X-ray scan image and a label, and the label is used to annotate the defect type of the metal plate X-ray scan image;

[0028] A pre-training module is used to put the image label pairs marked as clean labels into a clean subset that has been initialized to empty, and use the clean subset to pre-train two CNN models to obtain two CNN models with preliminary discrimination capabilities;

[0029] The formal training module is used to formally train two CNN models with preliminary discrimination capabilities using the training data set. In each round of formal training, image label pairs with clean labels are selected from the training data set and put into the clean subset. A clean subset is obtained in each round of formal training until the maximum number of training rounds is reached.

[0030] The clean sample set generation module is used to obtain the final clean sample set by finding the intersection of the v clean subsets obtained after the last v training rounds of formal training;

[0031] The classification module is used to train a reinitialized CNN model using a clean sample set, and use the trained CNN model to classify metal sheet defects.

[0032] The present invention also proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the above-mentioned method for detecting noise labels in metal plate radiographic images is implemented.

[0033] The present invention also provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the metal plate radiographic image noise label detection method as described above.

[0034] The beneficial effects of the present invention are as follows: the present invention can accurately and quickly detect noisy labels, obtain a clean label set, avoid the damage of noisy labels to model performance, save manual labeling costs, and facilitate the factory to perform defect detection on products of each metal plate processing production line of different materials. The present invention uses a large visual language model to assist in the screening of noisy labels, thereby improving the screening efficiency. When the present invention estimates the posterior probability of the cross loss, the cross loss of the two CNNs is comprehensively considered to further eliminate their self-confirmation bias. When the two CNNs are separately supervised for training, the drop out operation is used to randomly discard some neurons, so that the two CNNs produce inconsistent prediction results during training, thereby causing differences in the GMM-based division. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a flow chart of the metal plate radiographic image noise label detection method proposed by the present invention. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] Embodiment 1

[0038] The present invention proposes a method for detecting noise labels in metal plate radiographic images. The process of the method is as follows: Figure 1 As shown, the following steps are included:

[0039] S1. Using the industrial defect visual language model to mark each label in the training data set is a clean label or a noisy label; the training data set includes a plurality of image label pairs, each image label pair includes a metal plate X-ray scanning image and a label, and the label is used to mark the defect type of the metal plate X-ray scanning image; in S1, using the industrial defect visual language model to mark each label in the training data set is a clean label or a noisy label specifically as follows:

[0040] Set a prompt word for the industrial defect visual language model, and set the prompt word to "This is a X-ray scan image of a metal plate. The metal plate contains {the given label of the image Corresponding defect type name}? Please answer yes or no. "; The industrial defect visual language model used in this embodiment is AnomalyGPT. In addition, other visual language models can also be used.

[0041] The training data set Input the industrial defect visual language model, record the position of the image label pair with the answer "yes" as True, and the position of the image label pair with the answer "no" as False, where, is the i-th metal plate X-ray scanning image, is the i-th label, and n is the logarithm of the image label pair. We get a rough sequence of noise label detection results based on a large model. ,in Represents the labeling result of the current i-th image label pair.

[0042] Defect label detection result sequence The image label pairs marked as True are selected, and the labels in these image label pairs are used as clean labels.

[0043] S2. Put the image label pairs marked as clean labels into the clean subset that has been initialized to empty ,Use the clean subset to pre-train two CNN models, and obtain two CNN models with preliminary discrimination ability;

[0044] S3. Use the training data set to formally train two CNN models with preliminary discrimination capabilities. In each round of formal training, image label pairs with clean labels are selected from the training data set and put into the clean subset. A clean subset is obtained in each round of formal training; until the maximum number of training rounds is reached ; S3 specifically includes:

[0045] S3.1. At the beginning of the current training epoch, two CNN models with preliminary discriminative capabilities are used to calculate the training data set. The cross entropy loss of each pair of image labels in , and the loss sequence is obtained ;

[0046]

[0047] In the formula, is the i-th metal plate X-ray scanning image, is the i-th label, n is the number of image label pairs, It means that the cross entropy loss is calculated for the i-th image label pair using the first CNN model with preliminary discriminative ability. Indicates that the cross entropy loss is calculated for the i-th image label pair using the second CNN model with preliminary discriminative ability; represents the cross entropy loss of the i-th image label pair;

[0048] S3.2. Use a two-component Gaussian mixture model (GMM) to analyze the loss sequence Fitting is performed to estimate the probability that the label in all image label pairs is a clean label, and a probability sequence is obtained , where Represents the cross entropy loss of the Gaussian mixture model corresponding to the component g with the smallest mean for the i-th pair of image labels after fitting the distribution of the loss sequence Q The predicted probability of is the probability that the label in the i-th image-label pair is a clean label; here, different from the existing method that calculates the cross loss based on two CNNs respectively and uses two 2-component GMMs for posterior probability estimation, the present invention sums the cross entropy losses of the two CNNs to further eliminate their self-confirmation bias, and then only uses one 2-component GMM to estimate the posterior probability of the cross loss.

[0049] S3.3. Copy the clean subset generated by S2 , A clean subset labeled for industrial defect visual language model, is the clean subset of the current training round t, and based on the probability sequence W, selects the image label pairs whose probability of being a clean label is greater than the threshold and puts them into , the threshold of this embodiment is 0.5, and the clean subset of the current training round t is obtained ;

[0050] S3.4. Two CNN models with preliminary discrimination capabilities have the dropout function turned on, randomly discarding some neurons, so that the two CNNs produce inconsistent prediction results during training. Supervised training of two CNN models with preliminary discriminative capabilities. During training, the AutoAugment strategy is used to enhance the data of input image label pairs, and the training loss is the cross entropy loss.

[0051] S3.5. Turn off the drop out operation and increment the training round t by 1. If t does not reach the maximum training round , then jump to S3.1, otherwise execute S4.

[0052] S4, finding the intersection of the v clean subsets obtained after the last v training rounds of formal training to obtain the final clean sample set; S4 is specifically:

[0053]

[0054] In the formula, is the final clean sample set, is the i-th image label pair, is the i-th metal plate X-ray scanning image, is the i-th label, is the intersection of v clean subsets obtained after v training rounds, For the The clean subset obtained after training rounds; is the maximum number of training times, is the training data set.

[0055] S5. A reinitialized CNN model is trained using the clean sample set, and a basic cross loss is used to obtain satisfactory classification performance, thereby avoiding the damage of the noise label to the model performance. The trained CNN model is used for metal plate defect classification. The present invention reduces the cost of manual labeling and facilitates the factory to perform defect detection on products of each metal plate processing production line of different materials.

[0056] Embodiment 2

[0057] The present invention proposes a metal plate radiographic image noise label detection system, comprising:

[0058] An industrial defect visual language model is used to mark whether each label in a training data set is a clean label or a noise label; the training data set includes a plurality of image-label pairs, each image-label pair includes a metal plate X-ray scan image and a label, and the label is used to annotate the defect type of the metal plate X-ray scan image;

[0059] A pre-training module is used to put the image label pairs marked as clean labels into a clean subset that has been initialized to empty, and use the clean subset to pre-train two CNN models to obtain two CNN models with preliminary discrimination capabilities;

[0060] The formal training module is used to formally train two CNN models with preliminary discrimination capabilities using the training data set. In each round of formal training, image label pairs with clean labels are selected from the training data set and put into the clean subset. A clean subset is obtained in each round of formal training until the maximum number of training rounds is reached.

[0061] The clean sample set generation module is used to obtain the final clean sample set by finding the intersection of the v clean subsets obtained after the last v training rounds of formal training;

[0062] The classification module is used to train a reinitialized CNN model using a clean sample set, and use the trained CNN model to classify metal sheet defects.

[0063] The implementation methods of each module and module function in the system are completely consistent with the steps of the method in Example 1, so they will not be repeated here.

[0064] Embodiment 3

[0065] The present invention proposes an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting noise labels in metal plate radiographic images as described in the first embodiment is implemented.

[0066] Embodiment 4

[0067] The present invention provides a computer-readable storage medium storing a computer program, wherein the computer program enables a computer to execute the method for detecting noise labels in metal plate radiographic images as described in the first embodiment.

[0068] In the embodiments disclosed in the present application, the computer storage medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. The computer storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. More specific examples of computer storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0069] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0070] The above are only preferred embodiments of the present invention. The protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.

Claims

1. A method for detecting noise labels in metal plate radiographic images, characterized in that: The following steps are involved: S1. Using the industrial defect visual language model to mark each label in the training data set as a clean label or a noise label; the training data set includes a plurality of image-label pairs, each image-label pair includes a metal plate X-ray scan image and a label, and the label is used to mark the defect type of the metal plate X-ray scan image; S2, putting the image label pairs marked as clean labels into the clean subset that has been initialized to empty, and using the clean subset to pre-train two CNN models to obtain two CNN models with preliminary discrimination capabilities; S3. Use the training data set to formally train two CNN models with preliminary discrimination capabilities. In each round of formal training, image label pairs with clean labels are selected from the training data set and put into the clean subset. A clean subset is obtained in each round of formal training until the maximum number of training rounds is reached. S4, finding the intersection of the v clean subsets obtained after the last v training rounds of formal training to obtain the final clean sample set; S5. Use the clean sample set to train a reinitialized CNN model, and the trained CNN model is used for metal plate defect classification.

2. The method for detecting noise labels of metal plate radiographic images according to claim 1, characterized in that: In S1, the use of the industrial defect visual language model to mark each label in the training data set as a clean label or a noise label is specifically as follows: Set the prompt word for the industrial defect visual language model, and set the prompt word to "This is a X-ray scan image of a metal plate. The metal plate contains {the given label of the image Corresponding defect type name}? Please answer yes or no. "; The training data set Input the industrial defect visual language model and take the label in the image label pair with the answer "yes" as the clean label, where is the i-th metal plate X-ray scanning image, is the i-th label, and n is the number of image-label pairs.

3. The method for detecting noise labels of metal plate radiographic images according to claim 1, characterized in that: S3 specifically includes: S3.

1. At the beginning of the current training round, two CNN models with preliminary discrimination capabilities are used to calculate the training data set The cross entropy loss of each pair of image labels in , and the loss sequence is obtained ; ； In the formula, is the i-th metal plate X-ray scanning image, is the i-th label, n is the number of image label pairs, It means that the cross entropy loss is calculated for the i-th image label pair using the first CNN model with preliminary discriminative ability. Indicates that the cross entropy loss is calculated for the i-th image label pair using the second CNN model with preliminary discriminative ability; represents the cross entropy loss of the i-th image label pair; S3.

2. Use a two-component Gaussian mixture model to model the loss sequence Fitting is performed to estimate the probability that the label in all image label pairs is a clean label, and a probability sequence is obtained , where Represents the cross entropy loss of the Gaussian mixture model corresponding to the component g with the smallest mean for the i-th pair of image labels after fitting the distribution of the loss sequence Q The predicted probability of is the probability that the label in the i-th image label pair is a clean label; S3.

3. Copy the clean subset generated by S2 , A clean subset labeled for industrial defect visual language model, is the clean subset of the current training round t, and based on the probability sequence W, selects the image label pairs whose probability of being a clean label is greater than the threshold and puts them into , get the clean subset of the current training round t ; S3.

4. Two CNN models with preliminary discrimination capabilities turn on the dropout function, randomly discard some neurons, and use the clean subset Supervised training of two CNN models with preliminary discriminative capabilities. During training, the AutoAugment strategy is used to enhance the data of input image label pairs, and the training loss is the cross entropy loss. S3.

5. Turn off the drop out operation and increment the training round t by 1. If t does not reach the maximum training round , then jump to S3.1, otherwise execute S4.

4. The method for detecting noise labels of metal plate radiographic images according to claim 1, characterized in that: In S4, the intersection of the v clean subsets obtained after the last v training rounds of formal training is obtained to obtain the final clean sample set, which is expressed as follows: ； In the formula, is the final clean sample set, is the i-th image label pair, is the i-th metal plate X-ray scanning image, is the i-th label, is the intersection of v clean subsets obtained after v training rounds, For the The clean subset obtained after training rounds; is the maximum number of training times, is the training data set.

5. A metal plate radiographic image noise label detection system, characterized in that: include: Industrial Defect Visual Language Model, which is used to label whether each label in the training dataset is a clean label or a noisy label; The training data set includes a plurality of image-label pairs, each of which includes a metal plate X-ray scanning image and a label, wherein the label is used to annotate the defect type of the metal plate X-ray scanning image; A pre-training module is used to put the image label pairs marked as clean labels into a clean subset that has been initialized to empty, and use the clean subset to pre-train two CNN models to obtain two CNN models with preliminary discrimination capabilities; The formal training module is used to formally train two CNN models with preliminary discrimination capabilities using the training data set. In each round of formal training, image label pairs with clean labels are selected from the training data set and put into the clean subset. A clean subset is obtained in each round of formal training until the maximum number of training rounds is reached. The clean sample set generation module is used to obtain the final clean sample set by finding the intersection of the v clean subsets obtained after the last v training rounds of formal training; The classification module is used to train a reinitialized CNN model using a clean sample set, and use the trained CNN model to classify metal sheet defects.

6. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the method for detecting noise labels in metal plate radiographic images as described in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium storing a computer program, characterized in that: The computer program enables a computer to execute the method for detecting noise labels in metal plate radiographic images as described in any one of claims 1 to 4.

Images