Industrial defect detection method, device, equipment and readable storage medium

Abstract

The application discloses an industrial defect detection method, device, equipment and readable storage medium, comprising the steps of: acquiring a historical detection model of an original product, a historical training set corresponding to the historical detection model, and a positive sample set of a product to be detected; generating a target training set according to the positive sample set and the historical training set; establishing a comparative detection model, and training the comparative detection model according to the target training set to obtain a plurality of trained detection models; generating a target detection model according to a preset update strategy, the historical detection model and the trained detection model; and detecting the product to be detected by using the target detection model. The application improves the defect detection accuracy of a new product when a product is rapidly changed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to an industrial defect detection method, device and equipment and readable storage medium. BACKGROUND

[0002] With the development of deep learning neural networks, more and more are applied to industrial product appearance defect detection. However, the good performance of the deep learning model depends on a large number of training samples, and such high sample dependence leads to poor generalization ability of the model, resulting in a significant decline in performance when the existing model is applied to a new product. This limits the practical application of deep learning technology in industry, because in practical application, on the one hand, defect samples for training are often very rare and difficult to obtain, on the other hand, in general industrial scenarios, a production line will produce a variety of different specifications of products, and often needs to quickly switch products and models. Therefore, when deep learning technology is used in the rapid change type scene of industrial defect detection, there is only one product that can be detected, and the adaptability is poor. SUMMARY

[0003] The main purpose of the present application is to provide an industrial defect detection method, device and equipment and readable storage medium, which aims to solve the problem that only one product can be detected in the rapid change type scene of industrial defect detection, and the adaptability is poor.

[0004] To achieve the above purpose, the present application provides an industrial defect detection method, which comprises the following steps:

[0005] Obtaining a historical detection model of an original product, a historical training set corresponding to the historical detection model, and a positive sample set of a to-be-tested product;

[0006] Generating a target training set according to the positive sample set and the historical training set;

[0007] Establishing a comparison detection model, and training the comparison detection model according to the target training set to obtain a plurality of trained detection models;

[0008] Generating a target detection model according to a preset update strategy, the historical detection model and the trained detection model;

[0009] Detecting the to-be-tested product by using the target detection model.

[0010] Optionally, the step of selecting a target training set according to the positive sample set and the historical training set comprises:

[0011] Generating a plurality of pairs of positive samples according to the positive sample set, wherein each pair of positive samples comprises two images;

[0012] According to a preset composition ratio of the target training set and a number of the positive sample pairs, a plurality of negative sample pairs are selected from the historical training set;

[0013] Based on the positive sample pairs and the negative sample pairs, a target training set is generated.

[0014] Optionally, the historical training set comprises a plurality of pairs of pictures;

[0015] The step of selecting a plurality of negative sample pairs from the historical training set according to a preset composition ratio of the target training set and a number of the positive sample pairs comprises:

[0016] Histogram and gradient information of each picture in the historical training set is obtained;

[0017] The historical training set is clustered according to the histogram and the gradient information, and at least one category is obtained;

[0018] According to the histogram and the gradient information, a distance between each pair of pictures and a cluster center in each category is calculated;

[0019] According to a preset composition ratio and a number of the positive sample pairs, a plurality of negative sample pairs closest to the cluster center in each category are selected.

[0020] Optionally, the step of establishing a contrast detection model comprises:

[0021] An input pair of pictures is received, wherein the pair of pictures comprises a positive sample picture and a to-be-tested picture;

[0022] Feature maps of the positive sample picture and the to-be-tested picture at different scales are extracted respectively;

[0023] Feature cross contrast analysis is performed on the feature maps, and a feature difference matrix of the feature maps on each channel is obtained;

[0024] Channel merging is performed on the feature difference matrix, and a merged matrix is obtained;

[0025] Convolution operation is performed on the merged matrix, and a defect heat map of the to-be-tested picture is obtained.

[0026] Optionally, the feature maps comprise a to-be-tested feature map and a positive sample feature map;

[0027] The step of performing feature cross contrast analysis on the feature maps to obtain a feature difference matrix of the feature maps on each channel comprises:

[0028] The feature channel of the positive sample feature map and the feature channel of the to-be-detected feature map in the same scale and the adjacent scale are subtracted respectively to obtain a feature difference matrix of each channel of the feature map.

[0029] Optionally, the step of training the contrast detection model according to the target training set to obtain a plurality of trained detection models comprises:

[0030] The negative sample pictures in the target training set are labeled, and the contrast detection model is trained according to the target training set, and a plurality of training defect heat maps of the contrast detection model are output.

[0031] According to the negative sample picture and the training defect heat map, a loss function of the contrast detection model is calculated.

[0032] According to the loss function, the contrast detection model is trained to obtain a plurality of trained detection models.

[0033] Optionally, the step of generating a target detection model according to a preset update strategy, the historical detection model and the trained detection model comprises:

[0034] A plurality of trained detection models are verified by using a preset verification set, and a target trained detection model with the best performance is selected.

[0035] According to the historical detection model and the trained detection model, a target detection model is generated through a preset update strategy.

[0036] In addition, in order to achieve the above-mentioned purpose, the present application also provides an industrial defect detection device, which comprises:

[0037] A data acquisition module is configured to acquire a historical detection model of a raw product, a historical training set corresponding to the historical detection model, and a positive sample set of a to-be-detected product.

[0038] A training set generation module is configured to generate a target training set according to the positive sample set and the historical training set.

[0039] A model training module is configured to establish a contrast detection model, and train the contrast detection model according to the target training set to obtain a plurality of trained detection models.

[0040] A model generation module is configured to generate a target detection model according to a preset update strategy, the historical detection model and the trained detection model.

[0041] A defect detection module is configured to detect the to-be-detected product by using the target detection model.

[0042] In addition, to achieve the above object, the present application also provides an industrial defect detection device, comprising a memory, a processor and an industrial defect detection program stored on the memory and executable on the processor, which realizes the steps of the industrial defect detection method as described above when executed by the processor.

[0043] In addition, to achieve the above object, the present application also provides a computer readable storage medium, which stores an industrial defect detection program, which realizes the steps of the industrial defect detection method as described above when executed by a processor.

[0044] The present application provides an industrial defect detection method, device, equipment and readable storage medium, by acquiring a historical detection model of a original product, a historical training set corresponding to the historical detection model, and a positive sample set of a product to be detected, and generating a target training set according to the positive sample set and the historical training set, establishing a comparative detection model, and training the comparative detection model according to the target training set, a plurality of trained detection models are obtained, which can generate a training set by using the sample of the original product and a small amount of positive sample of the product to be detected, and then realize the training of the detection model, in the case that the image data of the product to be detected is limited, the difference between the sample of the original product and the sample to be detected of the product to be detected is learned, and the adaptability of the model is improved. By generating a target detection model according to a preset updating strategy, the historical detection model and the trained detection model, and detecting the product to be detected by using the target detection model, the already trained historical model can be fully utilized, the new comparative detection model can be quickly converged, the training time can be greatly shortened, and the defect detection of the new product can be realized in the scene of the rapid change of the production product, and the adaptability of the defect detection model to the defect detection of the new product is improved. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 is a structural schematic diagram of a hardware running environment related to an embodiment of the industrial defect detection method of the present application;

[0046] Figure 2 is a flowchart of an embodiment of the industrial defect detection method of the present application;

[0047] Figure 3 is a flowchart of step S20 in an embodiment of the industrial defect detection method of the present application;

[0048] Figure 4 is a schematic diagram of the framework structure of the comparative detection model of the industrial defect detection method of the present application;

[0049] Figure 5 is a flowchart of an embodiment of the industrial defect detection method of the present application;

[0050] Figure 6 is a flowchart of three embodiments of the industrial defect detection method of the present application.

[0051] The implementation, functional features and advantages of the present application will be further described with reference to the accompanying drawings and embodiments. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application more clear, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0053] As shown in Figure 1 , the terminal structure shown in Figure 1 is a schematic diagram of the terminal structure of the hardware running environment involved in the embodiment of the present application.

[0054] The terminal of the embodiment of the present application is an industrial defect detection device.

[0055] As shown in Figure 1 , the terminal can include a processor 1001 such as a CPU, a communication bus 1002, a user interface 1003, a DVI interface 1004, a USB interface 1005, and a memory 1006. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 can include a display screen (Display) and an input unit such as a keyboard (Keyboard). Optionally, the user interface 1003 can also include a standard wired interface and a wireless interface. The DVI interface 1004 can optionally include a standard wired interface and is connected to other external devices through a DVI line. The USB interface 1005 can optionally include a standard wired interface and is connected to other external devices through a USB connection line. The memory 1006 can be a high-speed RAM memory or a stable memory (non-volatile memory) such as a magnetic disk memory. The memory 1006 can optionally be a storage device for industrial defect detection independent of the aforementioned processor 1001.

[0056] Optionally, the terminal can also include an audio circuit and the like, which will not be described here.

[0057] Those skilled in the art can understand that Figure 1 the terminal structure shown in the above does not constitute a limitation on the terminal, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0058] As shown in Figure 1As shown, the memory 1006 as a computer storage medium can include an operating system, a DVI interface module, a USB interface module, a user interface module, and an industrial defect detection program.

[0059] In Figure 1 In the terminal shown, the DVI interface 1004 is mainly used to connect external devices and communicate data with the external devices; the USB interface 1005 is mainly used to connect external devices and communicate data with the external devices; the user interface 1003 is mainly used to connect clients and communicate data with the clients; and the processor 1001 can be used to call the industrial defect detection program stored in the memory 1006 and perform the following operations:

[0060] Obtain a historical detection model of a original product, a historical training set corresponding to the historical detection model, and a positive sample set of a product to be tested;

[0061] Generate a target training set according to the positive sample set and the historical training set;

[0062] Establish a comparison detection model, and train the comparison detection model according to the target training set to obtain a plurality of trained detection models;

[0063] Generate a target detection model according to a preset update strategy, the historical detection model, and the trained detection model;

[0064] Detect the product to be tested by using the target detection model.

[0065] Further, the processor 1001 can call the industrial defect detection program stored in the memory 1006 and further perform the following operations:

[0066] Generate a plurality of positive sample pairs according to the positive sample set, wherein each sample pair includes two images;

[0067] Select a plurality of negative sample pairs from the historical training set according to a preset composition ratio of the target training set and a number of the positive sample pairs;

[0068] Generate a target training set based on the positive sample pairs and the negative sample pairs.

[0069] Further, the processor 1001 can call the industrial defect detection program stored in the memory 1006 and further perform the following operations:

[0070] Obtain a histogram and gradient information of each image in the historical training set;

[0071] Cluster the historical training set according to the histogram and the gradient information to obtain at least one category;

[0072] According to the histogram and the gradient information, a distance between each pair of picture pairs and a cluster center is calculated in each category;

[0073] According to a preset composition ratio and a number of the positive sample pairs, a number of negative sample pairs closest to the cluster center in each category is selected.

[0074] Further, the processor 1001 can invoke an industrial defect detection program stored in the memory 1006, and further perform the following operations:

[0075] Receiving an input picture pair, wherein the picture pair includes a positive sample picture and a to-be-tested picture;

[0076] Respectively extracting feature maps of the positive sample picture and the to-be-tested picture under different scales;

[0077] Performing feature cross comparison analysis on the feature maps to obtain a feature difference matrix of the feature maps on each channel;

[0078] Performing channel merging on the feature difference matrix to obtain a merged matrix;

[0079] Performing convolution operation on the merged matrix to obtain a defect heat map of the to-be-tested picture.

[0080] Further, the processor 1001 can invoke an industrial defect detection program stored in the memory 1006, and further perform the following operations:

[0081] Respectively performing feature channel subtraction on the positive sample feature maps and the to-be-tested feature maps under the same scale and adjacent scales to respectively obtain a feature difference matrix of the feature maps on each channel.

[0082] Further, the processor 1001 can invoke an industrial defect detection program stored in the memory 1006, and further perform the following operations:

[0083] Labeling negative sample pictures in a target training set, and training the contrast detection model according to the target training set to output a plurality of training defect heat maps of the contrast detection model;

[0084] According to the negative sample pictures and the training defect heat maps, a loss function of the contrast detection model is calculated;

[0085] According to the loss function, the contrast detection model is trained to obtain a plurality of trained detection models.

[0086] Further, the processor 1001 can invoke an industrial defect detection program stored in the memory 1006, and further perform the following operations:

[0087] The preset verification set is used to verify a plurality of the trained detection models, and a target trained detection model with optimal performance is selected.

[0088] According to the historical detection model and the trained detection model, a target detection model is generated through a preset updating strategy.

[0089] The specific embodiments of the industrial defect detection equipment of the present application are basically the same as the following embodiments of the industrial defect detection program, and are not repeated here.

[0090] Please refer to Figure 1 , Figure 1 The flowchart of the first embodiment of the industrial defect detection method of the industrial defect detection equipment of the present application, the industrial defect detection method of the industrial defect detection equipment provided by the present embodiment comprises the following steps:

[0091] In step S10, the historical detection model of the original product, the historical training set corresponding to the historical detection model, and the positive sample set of the to-be-tested product are obtained.

[0092] The present application is mainly applied to the appearance defect detection of industrial products on the production line, and is suitable for the case that the appearance of the new product to be switched and the current original product is not particularly different, for example, products of the same series and different batches of the same product. In an industrial scenario, a production line can produce products of different specifications, and it is often necessary to quickly switch products and models during production. However, when performing appearance defect detection on a new product, the generalization ability of the historical detection model corresponding to the original product is poor, and the historical detection model cannot be directly applied to the new product for appearance defect detection. Therefore, in the present application, the historical detection model of the original product, the historical training set, and the positive sample set of the to-be-tested new product are obtained, the model is retrained, and the appearance defect detection of the original product and the new product is realized at the same time, thereby improving the adaptability of the model.

[0093] The historical detection model of the original product is the latest defect detection model corresponding to the original product. The historical training set is the training sample used when training the historical detection model, including negative sample pictures with defects and positive sample pictures without defects. The positive sample set of the to-be-tested product is a picture set without defects of the to-be-tested product.

[0094] In step S20, a target training set is generated according to the positive sample set and the historical training set.

[0095] The target training set exists in the form of a picture pair, including OK (positive sample picture) and OK (the to-be-tested picture has no defects), or OK and NG (the to-be-tested picture has defects). The number of picture pairs depends on the allowable model training time. Generally, the more the number of picture pairs, the higher the accuracy of the subsequently trained model.

[0096] In an embodiment, referring to Figure 2 , the step S20 further comprises:

[0097] Step S21, generating a plurality of pairs of positive samples according to the positive sample set, wherein each pair of positive samples comprises two images;

[0098] The positive sample set is an image without defects of a product to be tested, and a pair of positive samples can be formed by pairing two images. For example, if there are 20 images in the positive sample set, 10 pairs of positive samples can be formed by pairing two images each time. The pairing method can be random pairing combination, which is not limited in the present application.

[0099] Step S22, selecting a plurality of pairs of negative samples from the historical training set according to a preset composition ratio of the target training set and the number of the pairs of positive samples;

[0100] In an embodiment, the step S22 further comprises:

[0101] Step A221, obtaining a histogram and gradient information of each image in the historical training set;

[0102] For each image in the historical training set, a histogram is calculated. If the image is a color image, a color histogram is calculated, and a HOG gradient information map of a grayscale image corresponding to the color histogram is calculated. The histogram refers to a direction gradient histogram, and the gradient information refers to a gradient direction of the histogram.

[0103] Step A222, clustering the historical training set according to the histogram and the gradient information to obtain at least one category;

[0104] In the embodiment, the number of categories is approximately equal to the number of types of products contained in the historical training set. For example, if there are N types of products in the historical training set, the number of categories obtained by clustering is also N. The clustering method can be K-Means (K-means) clustering, K-Means++, bi-kmeans, kernel k-means, etc., which is not limited in the present application.

[0105] Step A223, calculating a distance between each pair of images and a cluster center in each category according to the histogram and the gradient information;

[0106] In the embodiment, the pairs of images are sorted according to the distances from the cluster centers. The distance is defined as

[0107] d ij =d h +a×d g , wherein d h is a distance calculated by using the histogram, and dg is a weight parameter. Wherein, the distance can be selected in different forms, such as the common Euclidean distance, or Hamming distance, Chebyshev distance, etc., which are not limited in the present application.

[0108] Step A224, according to the preset composition ratio and the number of positive sample pairs, a plurality of negative sample pairs closest to the cluster center in each category are selected.

[0109] The preset composition ratio refers to the ratio between the positive sample pairs and the negative sample pairs in the target training set. The number of positive sample pairs is the number of sample pairs generated in step S21. Since the sample pairs in each category have been sorted by distance in step A223, only a plurality of negative sample pairs need to be selected according to the preset composition ratio and the number of positive sample pairs. For example, assuming that the ratio of positive and negative samples in the target training set is 1:K. For each sample pair in step S21, the distance of the pair to the top K negative sample pairs of each class in the historical training set is calculated. Assuming that there are C clusters in the training set, there are CxK distances, corresponding to CxK training set sample pairs, and the top K smallest distance sample pairs are selected. Assuming that the number of sample pairs in step S21 is N, then NxK negative sample pairs are selected from the training set.

[0110] Step S23, generating a target training set based on the positive sample pairs and the negative sample pairs.

[0111] After obtaining the positive sample pairs and the negative sample pairs, the positive sample pairs and the negative sample pairs are recombined to generate a picture pair including one positive sample picture and one negative sample picture, or a picture pair including one positive sample picture and one negative sample picture, to form a target training set. That is, the target training set also exists in the form of a picture pair, and the picture pair at least includes one positive sample picture.

[0112] Step S30, establishing a contrast detection model and training the contrast detection model according to the target training set to obtain a plurality of trained detection models;

[0113] In the present embodiment, since the generalization ability of the deep learning model is weak and the data dependency is strong, and the image data collected for the new product is limited, it is difficult for the historical detection model to migrate to the new product with a small amount of data. Therefore, the present application proposes the idea of a contrast model, establishes a contrast detection model, and learns the difference between the to-be-detected sample and the sample of the original product through the target training set to improve the adaptability of the model. The target training set is trained on the contrast detection model, and a plurality of trained models can be trained, for details, please refer to the second and third embodiments, which are not described here.

[0114] Step S40, generating a target detection model according to a preset update strategy, the historical detection model and the trained detection model;

[0115] In an embodiment, the step S40 further comprises:

[0116] Step A41, verifying a plurality of the trained detection models by using a preset verification set, and selecting a target trained detection model with optimal performance;

[0117] In the embodiment, the preset verification set can be composed of pictures of new products without defects and pictures containing defects. The target trained detection model with optimal performance can be a model with the highest detection accuracy when the preset verification set is used for verification.

[0118] Step A42, generating a target detection model by a preset update strategy according to the historical detection model and the trained detection model.

[0119] The preset update strategy is represented as

[0120] θ t = aθ t-1 + (1-a)θ g

[0121] wherein θ t is the target detection model, θ t-1 is the target trained detection model, θ g is the historical detection model, and a is an adjustable parameter.

[0122] In the embodiment, by using the preset update strategy, the historical detection model that has been trained can be fully utilized to make the new model converge quickly and greatly shorten the training time.

[0123] Step S50, detecting the product to be tested by using the target detection model.

[0124] After obtaining the target detection model, the new product to be tested is detected by using the target detection model, and it is determined whether the product to be tested has appearance defects. It should be noted that the target detection model can be used to detect the original product or the new product, and the adaptability of the model in the product detection process is greatly improved.

[0125] This invention provides an industrial defect detection method. It acquires a historical detection model of the original product, a historical training set corresponding to the historical detection model, and a positive sample set of the product to be tested. Based on the positive sample set and the historical training set, a target training set is generated to establish a comparative detection model. The comparative detection model is then trained using the target training set to obtain multiple trained detection models. This method utilizes samples from the original product and a small number of positive samples from the product to be tested to generate a training set, thereby enabling the training of the detection model. Even with limited image data of the product to be tested, the method learns the differences between the samples of the product to be tested and those of the original product, improving the model's adaptability. By generating a target detection model based on a preset update strategy, the historical detection model, and the trained detection model, and using this target detection model to detect the product to be tested, the method fully utilizes the already trained historical model, allowing the new comparative detection model to converge quickly, significantly shortening the training time. This enables defect detection of new products in scenarios with rapid product changeovers, improving the defect detection model's adaptability to new product defect detection.

[0126] Further, please refer to Figure 3 Based on the first embodiment of the industrial defect detection method of the present invention, a second embodiment of the present invention is proposed, wherein the step of establishing a comparative detection model includes:

[0127] Step S31: Receive the input image pair, wherein the image pair includes a positive sample image and a test image;

[0128] In this embodiment, please refer to Figure 4 , Figure 4 The diagram shows the framework of the contrast detection model, which includes: an image input module, a feature extraction module, a channel feature subtraction module, a channel attention module, and a convolutional layer module.

[0129] The image input module is used to execute step S31. The positive sample image is a defect-free image of the product to be tested, which serves as the template image and will be referred to as image A in subsequent descriptions. The image to be tested can be a defect-free image or an image containing defects, and will be referred to as image B in subsequent descriptions. In other words, it is necessary to detect whether the image to be tested contains defects. It should be noted that the positive sample image and the image to be tested have the same size, for example, both are 256×256 pixels.

[0130] Step S32: Extract feature maps of the positive sample image and the image to be tested at different scales, respectively;

[0131] The feature extraction module is configured to perform step S32, wherein the feature maps include a feature map A corresponding to the picture A and a feature map B corresponding to the picture B, and the number of the feature maps is preferably 3. For example, the feature map A includes A1, A2 and A3 in descending order of scale, and the feature map B includes B1, B2 and B3 in descending order of scale, wherein A1 and B1, A2 and B2, and A3 and B3 have the same scale (length and width), and all have the same number of channels.

[0132] In step S33, feature cross comparison analysis is performed on the feature maps to obtain a feature difference matrix of each channel of the feature maps.

[0133] In an embodiment, step S33 further includes:

[0134] In step A331, feature channel subtraction is performed on the positive sample feature maps and the sample to be tested respectively at the same scale and adjacent scales to obtain a feature difference matrix of each channel of the feature maps.

[0135] In this embodiment, the channel feature subtraction module is configured to perform step S33, wherein the same scale refers to A1 and B1, A2 and B2, and A3 and B3, and the adjacent scale refers to A1 and B2, A2 and B3, A2 and B1, and A3 and B2. The feature maps [A1, B1], [A2, B2], [A3, B3], [A1, B2], [A2, B3], [A2, B1], and [A3, B2] are subjected to channel feature subtraction to obtain a feature difference matrix of each channel, which is denoted as CN in FIG. X. If the scale dimensions of the two pictures subjected to channel feature subtraction are different, the small-scale picture is expanded to be consistent with the large-scale picture through upsampling.

[0136] In step S34, channel merging is performed on the feature difference matrix to obtain a merged matrix.

[0137] The channel attention module is configured to perform step S34, wherein the method for performing channel merging can be a channel attention mechanism, and then the feature difference matrix of each branch is merged into a high-latitude matrix, i.e., all CNs in FIG. X are merged into a high-latitude matrix. It should be noted that the sizes of the feature difference matrices need to be the same for channel merging. If the latitude sizes of the branches are different, all the feature difference matrices need to be expanded to be consistent with the largest size through upsampling.

[0138] In step S35, convolution operation is performed on the merged matrix to obtain a defect heat map of the picture to be tested.

[0139] The convolutional layer module is used to perform step S35, and the matrix is combined through the convolutional layer and the like to continue combining the channels, adjust the size, and obtain the final single-channel heat map. The score of each pixel in the heat map represents the probability that the pixel is a defect, that is, it can be used to determine whether the to-be-tested picture has defects.

[0140] In the embodiment, the network model framework for comparison on multiple feature scales is proposed, which can ensure that the model learns to distinguish image features on a small number of new samples, and improve the accuracy of defect detection.

[0141] Further, referring to Figure 6 , the third embodiment of the present application is proposed based on the first embodiment of the industrial defect detection method of the present application, and the step of training the comparison detection model according to the target training set to obtain a plurality of trained detection models comprises:

[0142] Step S36, labeling the negative sample pictures in the target training set, and training the comparison detection model according to the target training set to output a plurality of training defect heat maps of the comparison detection model;

[0143] In the embodiment, the negative sample pictures can be labeled by a labeling tool, such as a make-sense labeling tool or a Labelme labeling tool, and the selection of the labeling tool is not limited in the present application, as long as the labeling of the negative sample pictures can be realized. After labeling is completed, the pictures in the target training set are input into the comparison detection model to obtain the training defect heat map.

[0144] Step S37, calculating the loss function of the comparison detection model according to the negative sample pictures and the training defect heat map;

[0145] The loss function is as follows:

[0146] L=NCC(d,y)+ce(edge(d),edge(y))+λ×L feature

[0147] Wherein, L is the loss value, d is the training defect heat map, y is the labeled negative sample picture, edge() is an edge extraction operation, ce() is cross entropy, and L feature is a feature-based Loss term, which is defined as follows:

[0148] For the six feature branches A1, A2, A3, B1, B2 and B3 of the second embodiment, the distances between A1 and B1, A2 and B2, and A2 and B2 are calculated, respectively, as L f1 , f2 and L f3. Because A comes from the positive sample set, it is always a positive sample (no defects), when B is also a positive sample, we hope that the three distances are small, when B has defects, we hope that the three distances become large. Therefore, when B is a positive sample, L feature = L f1 + L f2 + L f3 , when B is a defective sample, L feature = -1 * min (0, s (B)) (L f1 + L f2 + L f3 ), wherein s (B) represents the proportion of defective pixels in image B, and represents the severity of B containing defects, which can be calculated in advance.

[0149] Step S38, according to the loss function, the contrast detection model is trained to obtain a plurality of trained detection models.

[0150] After the loss value is calculated by the loss function, the contrast detection model updates each parameter in the model through back propagation to reduce the loss between the true value and the predicted value of the label map, so that the predicted value generated by the model approaches the true value, and then a plurality of trained detection models are obtained.

[0151] In this embodiment, by collecting a small amount of positive samples, combining similar defect samples extracted from the existing training set to train the model, and designing the loss function, the judgment accuracy of the model is improved, and the adaptability of the model to new products is improved.

[0152] In addition, the embodiment of the application also provides a computer readable storage medium, and the computer readable storage medium stores an industrial defect detection program, and the industrial defect detection program is executed by a processor to realize the following operations:

[0153] Obtain the historical detection model of the original product, the historical training set corresponding to the historical detection model, and the positive sample set of the to-be-tested product;

[0154] Generate a target training set according to the positive sample set and the historical training set;

[0155] Establish a contrast detection model, and train the contrast detection model according to the target training set to obtain a plurality of trained detection models;

[0156] Generate a target detection model according to a preset update strategy, the historical detection model and the trained detection model;

[0157] Detect the to-be-tested product by using the target detection model.

[0158] Further, the industrial defect detection program, when executed by the processor, further implements the following operations:

[0159] According to the positive sample set, a plurality of positive sample pairs are generated, and the sample pair includes two images;

[0160] According to a preset composition ratio of the target training set and the number of positive sample pairs, a plurality of negative sample pairs are selected from the historical training set;

[0161] Based on the positive sample pairs and the negative sample pairs, a target training set is generated.

[0162] Further, the industrial defect detection program, when executed by the processor, further implements the following operations:

[0163] Obtain the histogram and gradient information of each image in the historical training set;

[0164] According to the histogram and the gradient information, the historical training set is clustered to obtain at least one category;

[0165] According to the histogram and the gradient information, the distance between each pair of images and the cluster center in each category is calculated;

[0166] According to the preset composition ratio and the number of positive sample pairs, a plurality of negative sample pairs closest to the cluster center in each category are selected.

[0167] Further, the industrial defect detection program, when executed by the processor, further implements the following operations:

[0168] Receive an input image pair, wherein the image pair includes a positive sample image and a to-be-tested image;

[0169] Respectively extract feature maps of the positive sample image and the to-be-tested image under different scales;

[0170] Perform feature cross comparison analysis on the feature maps to obtain a feature difference matrix of the feature maps on each channel;

[0171] Merge the channels of the feature difference matrix to obtain a merged matrix;

[0172] Perform convolution operation on the merged matrix to obtain a defect heat map of the to-be-tested image.

[0173] Further, the industrial defect detection program, when executed by the processor, further implements the following operations:

[0174] Subtract the feature channels of the positive sample feature map and the to-be-tested feature map under the same scale and adjacent scales, respectively, to obtain a feature difference matrix of the feature maps on each channel.

[0175] Furthermore, when the industrial defect detection program is executed by the processor, it also performs the following operations:

[0176] The negative sample images in the target training set are labeled, and the contrast detection model is trained based on the target training set, and the heatmap of multiple training defects of the contrast detection model is output.

[0177] The loss function of the contrast detection model is calculated based on the negative sample images and the training defect heatmap.

[0178] Based on the loss function, the comparison detection model is trained to obtain multiple trained detection models.

[0179] Furthermore, when the industrial defect detection program is executed by the processor, it also performs the following operations:

[0180] Multiple trained detection models are validated using a preset validation set, and the target trained detection model with the best performance is selected.

[0181] Based on the historical detection model and the trained detection model, a target detection model is generated using a preset update strategy.

[0182] The present invention also proposes a computer-readable storage medium having a computer program stored thereon. The computer-readable storage medium may be... Figure 1 The memory 02 in the industrial defect detection equipment may also be at least one of ROM (Read-Only Memory) / RAM (Random Access Memory), magnetic disk, optical disk, etc. The computer-readable storage medium includes several information to enable the industrial defect detection equipment to perform the methods described in the various embodiments of the present invention.

[0183] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0184] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0185] Through the above description of the embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be realized by means of software and the necessary general hardware platform, of course, can also be realized by hardware, but in many cases, the former is a better embodiment.

[0186] The above are only preferred embodiments of the present application, and do not limit the patent scope of the present application, and any equivalent structure or equivalent process transformation made by using the content of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An industrial defect detection method characterized by, The industrial defect detection method comprises the steps of: obtaining a historical detection model of a raw product, a historical training set corresponding to the historical detection model, and a positive sample set of a product to be detected; generating a plurality of pairs of positive samples according to the positive sample set, wherein each pair of samples comprises two images; obtaining a histogram and gradient information of each image in the historical training set, wherein the histogram is a direction gradient histogram, and the gradient information is a gradient direction of the histogram; clustering the historical training set according to the histogram and the gradient information to obtain at least one category, wherein the historical training set comprises a plurality of pairs of images; calculating the distance between each pair of images and a cluster center in each category according to the histogram and the gradient information; selecting a plurality of negative sample pairs closest to the cluster center in each category according to a preset composition ratio and the number of positive sample pairs; generating a target training set based on the positive sample pairs and the negative sample pairs; establishing a comparative detection model and training the comparative detection model according to the target training set to obtain a plurality of trained detection models, wherein the step of establishing the comparative detection model comprises: receiving an input image pair, wherein the image pair comprises a positive sample image and a to-be-detected image; extracting feature maps of the positive sample image and the to-be-detected image at different scales, respectively, wherein the feature maps comprise to-be-detected feature maps and positive sample feature maps; subtracting the feature channels of the positive sample feature maps and the to-be-detected feature maps at the same scale and adjacent scales, respectively, to obtain a feature difference matrix of each channel of the feature maps; merging the channel of the feature difference matrix to obtain a merged matrix; performing convolution operation on the merged matrix to obtain a defect heat map of the to-be-detected image; generating a target detection model according to a preset update strategy, the historical detection model and the trained detection model; detecting the to-be-detected product by using the target detection model.

2. The method of industrial defect detection of claim 1, wherein, The step of training the comparative detection model according to the target training set to obtain a plurality of trained detection models comprises: annotating the negative sample images in the target training set, and training the comparative detection model according to the target training set to output a plurality of training defect heat maps of the comparative detection model; calculating a loss function of the comparative detection model according to the negative sample images and the training defect heat maps; training the comparative detection model according to the loss function to obtain a plurality of trained detection models.

3. The method of claim 2, wherein the step of determining the presence of the defect comprises the step of: The step of generating a target detection model according to a preset update strategy, the historical detection model and the trained detection model comprises: ​ verifying a plurality of trained detection models by using a preset verification set, and selecting a target trained detection model with the best performance; generating a target detection model by a preset update strategy according to the historical detection model and the trained detection model.

4. An industrial defect detection apparatus characterized by comprising: The industrial defect detection device comprises: a data acquisition module for obtaining a historical detection model of a raw product, a historical training set corresponding to the historical detection model, and a positive sample set of a product to be detected; The training set generation module is configured to generate a plurality of positive sample pairs from the positive sample set, the sample pairs including two images; obtain a histogram and gradient information of each image in the historical training set, wherein the histogram is a histogram of oriented gradients (HOG), and the gradient information is a gradient direction of the histogram; cluster the historical training set based on the histogram and the gradient information to obtain at least one category, wherein the historical training set includes a plurality of image pairs; calculate a distance between each image pair and a cluster center in each category based on the histogram and the gradient information; select a plurality of negative sample pairs closest to the cluster center in each category based on a preset composition ratio and a number of the positive sample pairs; and generate a target training set based on the positive sample pairs and the negative sample pairs. The model training module is configured to establish a contrast detection model, train the contrast detection model based on the target training set, and obtain a plurality of trained detection models. The establishment of the contrast detection model includes: receiving an input image pair, wherein the image pair includes a positive sample image and a to-be-detected image; extracting feature maps of the positive sample image and the to-be-detected image at different scales, wherein the feature maps include a to-be-detected feature map and a positive sample feature map; performing feature channel subtraction on the positive sample feature map and the to-be-detected feature map at the same scale and adjacent scales, respectively, to obtain a feature difference matrix of each channel of the feature maps; performing channel merging on the feature difference matrix to obtain a merged matrix; and performing convolution operation on the merged matrix to obtain a defect heat map of the to-be-detected image. The model generation module is configured to generate a target detection model based on a preset update strategy, the historical detection model, and the trained detection model. The defect detection module is configured to detect the to-be-detected product by using the target detection model.

5. An industrial defect detection apparatus characterized by comprising: The industrial defect detection device includes a memory, a processor, and an industrial defect detection program stored on the memory and executable on the processor. When the industrial defect detection program is executed by the processor, the steps of the industrial defect detection method according to any one of claims 1 to 3 are implemented.

6. A readable storage medium characterized by, The industrial defect detection program is stored on the readable storage medium. When the industrial defect detection program is executed by the processor, the steps of the industrial defect detection method according to any one of claims 1 to 3 are implemented.

Images