Industrial part defect detection method and device based on two-stage comparative learning

Abstract

The invention discloses an industrial part defect detection method and device based on two-stage comparative learning. The method comprises the following steps: acquiring an image data set of an industrial part; generating diversified synthesis abnormal samples; obtaining a discrimination network with a coarse granularity anomaly positioning capability through the training of the first stage, wherein the discrimination network is used for assisting the training process of the feature extraction network; in the second stage of training, a comparative learning framework is used for completing adaptive fine tuning of the feature extraction network, and the similarity between normal samples and the difference between the normal samples and synthesis abnormal samples are learned; and constructing a compression feature memory bank only containing normal features, and performing defect detection on the to-be-detected industrial part based on the compression feature memory bank and the trained defect detection model. Based on a two-stage contrast learning mode, the correlation between each normal industrial part and the difference between the normal part and the abnormal part are considered, and the defect detection precision is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of image defect detection, and in particular relates to an industrial parts defect detection method and device based on contrast learning and adversarial learning. Background Art

[0002] The purpose of image defect detection is to detect defects in objects in images, such as identifying defective products in the quality inspection process of the production process, thereby improving the yield rate. In recent years, with the growing demand for agricultural and industrial development, people's attention to this field has also risen sharply. Most existing methods attempt to use pre-trained models to extract features, thereby training classifiers or discriminant networks to identify defects. However, it is challenging to accurately identify all defect situations. This is mainly because the pre-trained models are trained for specific classification or segmentation tasks, and there is a domain gap between them and the defect detection image data, resulting in the frozen pre-trained model being unable to distinguish between normal features and abnormal features. For example, semantic segmentation and defect detection are different tasks, and the segmentation principles of the two are also different. Anomaly detection is an unsupervised learning method, and data that deviates from the normal distribution will be regarded as abnormal data. Therefore, in order to make the network adapt to the defect detection image and reduce the domain gap between the defect detection image data, the present invention uses knowledge such as fine-tuning learning, contrastive learning, and domain adaptation to research and design a defect detection method that can be adapted to defect detection image data. Because the defect samples in the data are scarce and their abnormal conditions are uncertain, the industrial parts image will be taken as an example to specifically introduce the designed industrial parts defect detection method. Summary of the invention

[0003] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide an industrial parts defect detection method and device based on two-stage contrastive learning. By synthesizing random anomalies, discriminant network pre-training, negative guided contrastive learning, and building a compressed feature memory library, the model's adaptability to industrial images is improved, thereby improving the accuracy of defect recognition.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions:

[0005] In a first aspect, the present invention provides an industrial parts defect detection method based on two-stage contrastive learning, comprising the following steps:

[0006] Collecting image data sets of industrial parts; the image data sets include training data sets and test data sets;

[0007] Generate abnormal data from normal sample images in the training data set to obtain diverse synthetic abnormal samples;

[0008] The normal samples and synthetic abnormal samples are input into a pre-established defect detection model for two-stage training, and a discriminant network with coarse-grained abnormality positioning capability is obtained through the first-stage training, and the discriminant network is used to assist the training process of the feature extraction network;

[0009] The data after the first stage of training is used for the second stage of training, and the contrastive learning framework is used to complete the adaptive fine-tuning of the feature extraction network, learn the similarities between normal samples and the differences between normal samples and synthetic abnormal samples; after the training is completed, a feature extraction network F that has been fine-tuned through the two-stage training process is obtained;

[0010] A compressed feature memory library containing only normal features is constructed, and defect detection is performed on the industrial parts to be inspected based on the compressed feature memory library and the trained defect detection model.

[0011] As a preferred technical solution, the image data set of industrial parts is collected, specifically:

[0012] Industrial cameras are used to complete the image data collection of industrial parts, and the collected image data sets are normalized and brightness uniformized.

[0013] As a preferred technical solution, the normal sample images in the training data set are used to generate abnormal data to obtain diverse synthetic abnormal samples, specifically:

[0014] Use a random seed seed to generate a two-dimensional Perlin noise mask P;

[0015] Using an additional texture image dataset, a random seed is used to select an image as the data source A of the abnormal area;

[0016] From the training data set T train Randomly select a normal sample image I∈T train , randomly apply a slight rotation less than or equal to the set angle and a rectangular rotation to the normal sample image to obtain an enhanced normal sample image I R ;

[0017] According to the generated Perlin noise mask, the abnormal data source image is cut and pasted to the normal sample image after data enhancement, thereby obtaining the synthesized abnormal image I G And its corresponding pixel-level label y G .

[0018] As a preferred technical solution, the first stage of training is used to obtain a discriminant network with coarse-grained anomaly location capability, specifically:

[0019] Use the image anomaly generation module G to build a synthetic anomaly dataset T G :

[0020]

[0021] Where i = {1,…,N} represents the index of the image in the training set;

[0022] A pre-trained feature extraction network is used to extract the features of the image. At the same time, a network symmetrical to the feature extraction network is constructed as the discriminant network. When the image enters the feature extraction network and the discriminant network, each layer of the network can obtain the feature map of the image, and the size of each layer of the feature map obtained by the feature extraction network corresponds to the size of each layer of the feature map obtained by the discriminant network. In the discriminant network, the feature map of each layer is first upsampled, and then the feature fusion is completed through jump connection. After being spliced ​​with the feature map of the corresponding size in the feature extraction network, the local context information of the feature map is obtained after convolution, thereby capturing the details and semantic information of the image on the features of different layers. During the training process, the weights of the feature extraction network are frozen, and only synthetic abnormal images are used to train the discriminant network.

[0023] Using synthetic abnormal image I G and the corresponding pixel-level label y G To train the discriminant network to obtain the coarse-grained anomaly location capability; when the synthetic abnormal image I G After being input into the network, a predicted feature map is output after passing through the feature extraction network and the discriminant network. Use the output and label to first calculate the loss function to update the learnable parameters of the discriminant network; in order to alleviate the imbalance between normal samples and abnormal samples, use focal loss as the first loss function L for discriminant network training dis , specifically:

[0024]

[0025] Among them, α t is a scaling factor associated with category t, γ is an adjustable parameter, and p t Corresponding to the predicted pixel classification, when p t =1 indicates abnormal category, p t =0 indicates the normal category.

[0026] As a preferred technical solution, the data after the first stage training is subjected to the second stage training, and the contrastive learning framework is used to complete the adaptive fine-tuning of the feature extraction network, and learn the similarities between normal samples and the differences between normal samples and synthetic abnormal samples, specifically:

[0027] Perform data augmentation on normal samples and obtain two enhanced views v 1 and v 2 ;

[0028] Construct a contrastive learning network C with two branches, one of which contains only a projection head, and the other contains a projection head and a prediction head, and the two branches share a feature extraction network; 1 and v 2 The two branches are sent to each other respectively, and the features of the last two layers are extracted and then spliced ​​and fused to obtain the projection head output p1 and the prediction head output z2. The one-way cosine similarity loss function value is calculated for the two outputs:

[0029]

[0030] Among them, |·|2 represents the Euclidean distance of the output features of the two branches; similarly, the projection head output p2 and the prediction head output z1 are obtained, and the calculation is Calculate the average bidirectional similarity to maintain balance:

[0031]

[0032] Among them, the "stop gradient" operation SG is used for z1 and z2 to prevent the collapse problem in the contrastive learning process; when M different data augmentation views are generated for each normal sample, the symmetric similarity loss function is as follows:

[0033]

[0034] At the same time, after the synthetic abnormal samples pass through the feature extraction network and the frozen discriminant network, the output is the predicted feature map From this, the focal loss is calculated as the negative sample guided loss function:

[0035]

[0036] The overall loss function of the second stage is:

[0037] L ncl =λ·L sym +(1-λ)·L neg

[0038] Among them, λ is a hyperparameter that balances the two loss functions.

[0039] As a preferred technical solution, the data of the normal sample is enhanced to obtain two enhanced views v 1 and v 2 , specifically:

[0040] After the positive sample enhancement module S, two different data enhancement views v are generated 1 and v 2 , in the positive sample enhancement module S

[0041] In the algorithm, the image is first cropped at random positions to obtain image blocks at different positions of the image; secondly, Gaussian filtering is performed on these image blocks to make specific features present in different states; finally, these image blocks are randomly flipped horizontally to reduce the model's dependence on the image direction and alleviate directional bias.

[0042] As a preferred technical solution, the construction of a compressed feature memory library containing only normal features is specifically as follows:

[0043] In the inference phase, the fine-tuned feature extraction network F is used as the backbone network, and the input is from the test dataset T test The test image I t ∈T test , extract adaptive features from the second and third layers, and combine them to obtain the fusion features of the image, thereby obtaining both shallow texture information and deep semantic content; then, these fusion features are used to construct the original memory library M O ;

[0044] A greedy core set sampling algorithm is used to identify the most representative features for constructing the compressed feature memory library M. * ;

[0045] Get the compressed feature memory library M * After that, the feature subset c closest to the image feature is sampled. * , calculate the maximum Euclidean distance between the image feature and the nearest feature in the feature subset to obtain the pixel-level anomaly score s of the test image t ; Since the pixel-level anomaly score is a feature map, the maximum value of the pixel-level anomaly scores is used as the image-level anomaly score.

[0046] In a second aspect, the present invention provides an industrial parts defect detection system based on two-stage contrastive learning, which is applied to the industrial parts defect detection method based on two-stage contrastive learning, including an image acquisition module, an image anomaly generation module, a discriminant network pre-training module, a negative guided contrastive learning module, and a compressed feature memory library construction module;

[0047] The image acquisition module is used to acquire an image data set of industrial parts; the image data set includes a training data set and a test data set;

[0048] The image anomaly generation module is used to generate abnormal data from normal sample images in the training data set to obtain diverse synthetic abnormal samples;

[0049] The discriminant network pre-training module is used to input the normal samples and synthetic abnormal samples into a pre-established defect detection model for two-stage training, and obtain a discriminant network with coarse-grained abnormality positioning capability through the first-stage training. The discriminant network is used to assist the training process of the feature extraction network;

[0050] The negative guided contrastive learning module is used to perform second-stage training on the data after the first-stage training, and use the contrastive learning framework to complete the adaptive fine-tuning of the feature extraction network, and learn the similarities between normal samples and the differences between normal samples and synthetic abnormal samples;

[0051] The compressed feature memory library construction module is used to construct a compressed feature memory library containing only normal features, and perform defect detection on the industrial parts to be inspected based on the compressed feature memory library and the trained defect detection model.

[0052] In a third aspect, the present invention provides an electronic device, the electronic device comprising:

[0053] at least one processor; and,

[0054] a memory communicatively connected to the at least one processor; wherein,

[0055] The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the industrial parts defect detection method based on two-stage contrast learning.

[0056] In a fourth aspect, the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the industrial parts defect detection method based on two-stage comparative learning.

[0057] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0058] 1. The present invention designs an image defect synthesis module and samples highly random Perlin noise to generate defect samples. The diversity of synthesized abnormal samples helps to make the potential space of positive samples more compact, thereby allowing a potential boundary to exist between normal samples and abnormal samples.

[0059] 2. The present invention designs a two-stage training framework for fine-tuning the feature extraction network, which includes a discriminant network pre-training framework and a negative guided contrastive learning framework. The former, as the first stage training process, aims to obtain a discriminant network with coarse-grained anomaly location capability, which is used to assist the training process of the feature extraction network; the latter inputs the pre-processed normal and abnormal image data into the defect detection model for the second stage training, and uses the contrastive learning framework to complete the adaptive fine-tuning of the feature extraction network, learning the similarities between normal samples and the differences between normal samples and synthetic abnormal samples. Ultimately, a feature extraction network adapted to the target industrial data set can be obtained, reducing the field gap with the target industrial data set.

[0060] 3. The present invention designs a compressed feature memory library construction framework, uses the fine-tuned feature extraction network, and constructs a compressed feature memory library containing only normal features through a greedy core set sampling algorithm, which is ultimately used for anomaly location reasoning. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0062] Figure 1 It is a flow chart of an industrial parts defect detection method based on two-stage comparative learning according to an embodiment of the present invention;

[0063] Figure 2 This is a flow chart of an image defect generation module according to an embodiment of the present invention;

[0064] Figure 3 A flowchart of a network pre-training framework for determining an embodiment of the present invention;

[0065] Figure 4 It is a flowchart of a negative guided contrastive learning framework according to an embodiment of the present invention;

[0066] Figure 5 A schematic diagram of a comparative learning framework according to an embodiment of the present invention;

[0067] Figure 6 A flowchart of a framework for building a compressed feature memory library according to an embodiment of the present invention;

[0068] Figure 7 A block diagram of an industrial parts defect detection method based on two-stage contrast learning according to an embodiment of the present invention;

[0069] Figure 8 It is a schematic diagram of the structure of the electronic device of the present invention. DETAILED DESCRIPTION

[0070] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0071] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0072] See also Figure 1 In one embodiment of the present application, a method for industrial parts defect detection based on two-stage contrastive learning is provided, comprising the following steps:

[0073] S1. Use industrial cameras to complete the image data collection of industrial parts, and collect images of normal samples for training and images of samples to be tested. After the image acquisition module, it is normalized and the brightness is uniformed, and then converted into a form that can be recognized by the model.

[0074] Furthermore, the industrial camera is used to complete the image data collection of industrial parts, and the images of normal samples for training and the images of samples to be tested are collected respectively. The area array camera is fixed in an overhead shooting position, and the samples are illuminated with uniform light to collect bright and uniformly illuminated industrial parts images. After the image acquisition module, it is normalized and the brightness is uniformed, and then converted into a form that can be recognized by the model.

[0075] S2. Please refer to Figure 2 , the normal sample images in the training data are used to generate abnormal data, thereby generating diverse synthetic abnormal samples. Training the network with such samples helps to make the feature space of normal samples more compact.

[0076] Furthermore, the normal sample images used for training are input into the image anomaly generation module G to generate abnormal data, thereby generating a variety of synthetic abnormal samples. Using such samples to train the network helps to make the feature space of normal samples more compact. The generation process includes generating random Perlin noise, pasting abnormal data sources, image rotation, color jittering, and abnormal label generation, specifically:

[0077] S21, using a random seed seed, generating a two-dimensional Perlin noise mask P;

[0078] S22, using an additional texture image dataset, using a random seed to select an image as the data source A of the abnormal area;

[0079] S23, from the training data set T train Randomly select a normal sample image I∈T train , where the image size is H×W×3, and a slight rotation of less than or equal to 5 degrees and a rectangular rotation are randomly applied to the image to obtain an enhanced normal sample image I R ;

[0080] S24, according to the generated Perlin noise mask, cut and paste the abnormal data source image to the normal sample image after data enhancement, thereby obtaining a synthesized abnormal image I G And its corresponding pixel-level label y G ∈{0,1} H×W×C .

[0081] S3. Please refer to Figure 3 ,Through the first stage of training process, a discriminant network with ,coarse-grained anomaly location capability is obtained to assist the ,training process of feature extraction network.

[0082] Furthermore, the first stage of training is called the discriminant network pre-training framework DNP, and its goal is to obtain a discriminant network with coarse-grained anomaly localization capability to assist the training process of the feature extraction network, specifically:

[0083] S31. First, use the image anomaly generation module G to construct a synthetic anomaly dataset:

[0084]

[0085] Wherein, i={1,…,N} represents the index of the image in the training set.

[0086] S32. In terms of feature extraction, we use the pre-trained feature extraction network WideResNet50 to extract the features of the image, relying on its powerful feature extraction capabilities. In terms of the discriminant network, we built a network symmetrical to WideResNet50 as the discriminant network. When the image enters the feature extraction network and the discriminant network, each layer of the network can obtain the feature map of the image, and the size of each layer of the feature map obtained by the feature extraction network corresponds to the size of each layer of the feature map obtained by the discriminant network. In the discriminant network, the feature map of each layer is first upsampled, and then the feature fusion is completed through jump connection. After being spliced ​​with the feature map of the corresponding size in the feature extraction network, the local context information of the feature map is obtained after convolution, which helps the network capture the details and semantic information of the image on the features of different layers. During the training process, we froze the weights of the feature extraction network and only used synthetic abnormal images to train the discriminant network.

[0087] S33, using synthetic abnormal image I G and the corresponding pixel-level label y G To train the discriminant network to obtain the coarse-grained anomaly location capability. G After being input into the network, a predicted feature map is output after passing through the feature extraction network and the discriminant network. The loss function is calculated using the output and label to update the learnable parameters of the discriminant network. In order to alleviate the imbalance between normal samples and abnormal samples, focal loss is used as the loss function L for discriminant network training. dis , specifically:

[0088]

[0089] Among them, α t is a scaling factor associated with category t, γ is an adjustable parameter, and p t Corresponding to the predicted pixel classification, when p t =1 indicates abnormal category, p t = 0 indicates the normal category. When the first stage of training is completed, we can obtain a discriminant network D with coarse-grained anomaly localization capability.

[0090] S4. Please refer to Figure 4 , Figure 5 ,The preprocessed normal and abnormal image data are input into the ,defect detection model for the second stage of training, and the ,contrastive learning framework is used to complete the adaptive fine-tuning of the ,feature extraction network, learning the similarities between normal samples and the ,differences between normal samples and synthetic abnormal samples.

[0091] Furthermore, the preprocessed normal and abnormal image data are input into the defect detection model for the second stage of training, and the contrastive learning framework is used to complete the adaptive fine-tuning of the feature extraction network, which is called the negative guided contrastive learning framework NCL, specifically:

[0092] S41, take the normal image I∈T train , respectively construct the training data of synthetic abnormal samples and normal samples. When constructing synthetic abnormal sample data, we use the anomaly generation module G to generate synthetic abnormal sample data I′ of I G and the corresponding label y′ G ∈{0,1} H×W×C :

[0093] I′ G =G(I)

[0094] After the positive sample enhancement module S, M different data enhancement views v are generated 1 ,…,v M In the positive sample enhancement module S, we first crop the image at random positions with a scaling factor of 0.2 to 1.0 to obtain image patches at different positions of the image; secondly, we apply Gaussian filtering with a standard deviation of σ = 0.1 to 2.0 on these image patches to make specific features present different states; finally, we apply random horizontal flipping with a probability of p = 0.5 to these image patches to reduce the model's dependence on image direction and alleviate directional bias. We set M = 2 and generate two enhanced views v for the normal image each time. 1 and v 2 .

[0095] S42. Construct a contrastive learning network C with two branches, one of which contains only one projection head, and the other contains one projection head and one prediction head, and the two branches share a feature extraction network. 1 and v 2 The two branches are sent to each other respectively, and the features of the last two layers are extracted and then spliced ​​and fused to obtain the projection head output p1 and the prediction head output z2. The one-way cosine similarity loss function value is calculated for the two outputs:

[0096]

[0097] Among them, |·|2 represents the Euclidean distance of the output features of the two branches. Similarly, we can get the projection head output p2 and the prediction head output z1, and calculate Calculate the average bidirectional similarity to maintain balance:

[0098]

[0099] Among them, the "stop gradient" operation SG is used for z1 and z2 to prevent the collapse problem in the contrastive learning process. When we generate M different data augmentation views for each normal sample, the symmetric similarity loss function is as follows:

[0100]

[0101] S43. At the same time, after the synthetic abnormal sample passes through the feature extraction network and the frozen discriminant network, the output is the predicted feature map From this, the focal loss is calculated as the negative sample guided loss function:

[0102]

[0103] S44. The overall loss function of the second stage is:

[0104] L ncl =λ·L sym +(1-λ)·L neg

[0105] Among them, λ is a hyperparameter that balances the two loss functions and is set to 0.5 by default. After the training is completed, we obtain a feature extraction network F that has been fine-tuned through a two-stage training process.

[0106] S5. Please refer to Figure 6 ,Through the greedy core set sampling algorithm, a compressed feature memory library containing only normal features is constructed for ,reasoning of anomaly location.

[0107] Furthermore, the construction of a compressed feature memory library containing only normal features for the reasoning of abnormal location is specifically as follows:

[0108] S51. In the inference phase, we use the fine-tuned feature extraction network F as the backbone network and input the test dataset T test The test image I t ∈T test , extract adaptive features from the second and third layers, and concatenate and fuse them to obtain the fusion features of the image, thereby obtaining both shallow texture information and deep semantic content. Then, these features are used to construct the original memory library M O .

[0109] S52. In order to minimize storage and computational overhead, we use a greedy core set sampling algorithm to identify the most representative features for constructing the compressed feature memory library M * :

[0110]

[0111] S53, obtain the compressed feature memory library M * After that, we sample the feature subset c closest to the image feature * , calculate the maximum Euclidean distance between the image feature and the nearest feature in the feature subset to obtain the pixel-level anomaly score s of the test image t :

[0112]

[0113] Where E(·) represents the Euclidean distance, N b (c * ) represents the feature subset c * Since the pixel-level anomaly score is a feature map, the maximum value of the pixel-level anomaly scores is used as the image-level anomaly score.

[0114] Based on the same idea as the industrial parts defect detection method based on two-stage contrastive learning in the above-mentioned embodiment, the present invention also provides an industrial parts defect detection system based on two-stage contrastive learning, which can be used to execute the above-mentioned industrial parts defect detection method based on two-stage contrastive learning. For ease of explanation, the structural schematic diagram of the embodiment of the industrial parts defect detection method based on two-stage contrastive learning only shows the parts related to the embodiment of the present invention. Those skilled in the art can understand that the illustrated structure does not constitute a limitation on the device, and may include more or fewer components than shown in the diagram, or combine certain components, or arrange the components differently.

[0115] See also Figure 7 In another embodiment of the present application, there is provided an industrial parts defect detection system 100 based on two-stage contrastive learning, the system comprising an image acquisition module 101, an image defect generation module 102, a discriminant network pre-training module 103, a negative guided contrastive learning module 104, and a compressed feature memory library construction module 105;

[0116] The image acquisition module 101 is used to acquire image data sets of industrial parts; the image data sets include training data sets and test data sets;

[0117] The image anomaly generating module 102 is used to generate abnormal data from normal sample images in the training data set to obtain diverse synthetic abnormal samples;

[0118] The discriminant network pre-training module 103 is used to input the normal samples and synthetic abnormal samples into a pre-established defect detection model for two-stage training, and obtain a discriminant network with coarse-grained abnormality positioning capability through the first-stage training. The discriminant network is used to assist the training process of the feature extraction network;

[0119] The negative guided contrastive learning module 104 is used to perform the second stage training on the data after the first stage training, and use the contrastive learning framework to complete the adaptive fine-tuning of the feature extraction network, and learn the similarities between normal samples and the differences between normal samples and synthetic abnormal samples;

[0120] The compressed feature memory library construction module 105 is used to construct a compressed feature memory library containing only normal features, and perform defect detection on the industrial parts to be inspected based on the compressed feature memory library and the trained defect detection model.

[0121] This application designs a two-stage comparative learning framework for the industrial parts defect detection algorithm. The biggest advantage of this model is its high model detection accuracy, low training consumption, and easy deployment.

[0122] In order to improve the efficiency and accuracy of industrial parts defect detection, the present invention generates abnormal data from normal sample images in training data, thereby generating diverse synthetic abnormal samples. Using such samples to train a network helps to make the feature space of normal samples more compact. Through the first-stage training process, a discriminant network with coarse-grained abnormality positioning capability is obtained, which is used to assist the training process of the feature extraction network. The preprocessed normal and abnormal image data are input into the defect detection model for the second-stage training, and a contrastive learning framework is used to complete the adaptive fine-tuning of the feature extraction network, and learn the similarities between normal samples and the differences between normal samples and synthetic abnormal samples. Through the greedy core set sampling algorithm, a compressed feature memory library containing only normal features is constructed for the reasoning of abnormality positioning.

[0123] It should be noted that the industrial parts defect detection system based on two-stage contrastive learning of the present invention corresponds one-to-one to the industrial parts defect detection method based on two-stage contrastive learning of the present invention. The technical features and beneficial effects described in the embodiment of the above-mentioned industrial parts defect detection method based on two-stage contrastive learning are applicable to the embodiment of the industrial parts defect detection system based on two-stage contrastive learning. For specific contents, please refer to the description in the embodiment of the method of the present invention, which will not be repeated here. This is hereby declared.

[0124] In addition, in the implementation of the industrial parts defect detection system based on two-stage contrastive learning in the above-mentioned embodiment, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation. That is, the internal structure of the industrial parts defect detection system based on two-stage contrastive learning is divided into different program modules to complete all or part of the functions described above.

[0125] See also Figure 8 In one embodiment, an electronic device for implementing an industrial parts defect detection method based on two-stage comparative learning is provided. The electronic device 200 may include a first processor 201, a first memory 202 and a bus, and may also include a computer program stored in the first memory 202 and executable on the first processor 201, such as an industrial parts defect detection program 203 based on two-stage comparative learning.

[0126] Among them, the first memory 202 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the first memory 202 can be an internal storage unit of the electronic device 200, such as a mobile hard disk of the electronic device 200. In other embodiments, the first memory 202 can also be an external storage device of the electronic device 200, such as a plug-in mobile hard disk, a smart memory card (SmartMediaCard, SMC), a secure digital (SecureDigital, SD) card, a flash card (FlashCard), etc. equipped on the electronic device 200. Further, the first memory 202 can also include both an internal storage unit of the electronic device 200 and an external storage device. The first memory 202 can not only be used to store application software and various types of data installed in the electronic device 200, such as the code of the industrial parts defect detection program 203 based on two-stage comparative learning, but also can be used to temporarily store data that has been output or is to be output.

[0127] In some embodiments, the first processor 201 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The first processor 201 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes various functions and processes data of the electronic device 200 by running or executing programs or modules stored in the first memory 202, and calling data stored in the first memory 202.

[0128] Figure 8 Only an electronic device with components is shown, and those skilled in the art will understand that Figure 7The structure shown does not constitute a limitation on the electronic device 200, and may include fewer or more components than shown, or a combination of certain components, or a different arrangement of components. The industrial parts defect detection program 203 based on two-stage contrastive learning stored in the first memory 202 in the electronic device 200 is a combination of multiple instructions, which can be implemented when running in the first processor 201:

[0129] S1. Collecting image datasets of industrial parts; the image datasets include training datasets and test datasets;

[0130] S2, generating abnormal data from normal sample images in the training data set to obtain diverse synthetic abnormal samples;

[0131] S3, inputting the normal samples and synthetic abnormal samples into a pre-established defect detection model for two-stage training, obtaining a discriminant network with coarse-grained abnormality location capability through the first-stage training, and the discriminant network is used to assist the training process of the feature extraction network;

[0132] S4, the data after the first stage training is trained in the second stage, and the contrastive learning framework is used to complete the adaptive fine-tuning of the feature extraction network, learn the similarities between normal samples and the differences between normal samples and synthetic abnormal samples; after the training is completed, a feature extraction network F that has been fine-tuned through the two-stage training process is obtained;

[0133] S5. Build a compressed feature memory library containing only normal features, and perform defect detection on the industrial parts to be inspected based on the compressed feature memory library and the trained defect detection model.

[0134] Furthermore, if the module / unit integrated in the electronic device 200 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a non-volatile computer-readable storage medium. The computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0135] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0136] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0137] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. An industrial parts defect detection method based on two-stage contrastive learning, characterized in that: The steps include: Collecting image data sets of industrial parts; the image data sets include training data sets and test data sets; Generate abnormal data from normal sample images in the training data set to obtain diverse synthetic abnormal samples; The normal samples and synthetic abnormal samples are input into a pre-established defect detection model for two-stage training, and a discriminant network with coarse-grained abnormality positioning capability is obtained through the first-stage training, and the discriminant network is used to assist the training process of the feature extraction network; The data after the first phase of training is used for the second phase of training, and the contrastive learning framework is used to complete the adaptive fine-tuning of the feature extraction network, learn the similarities between normal samples and the differences between normal samples and synthetic abnormal samples; After the training is completed, a feature extraction network F is obtained which has been fine-tuned through a two-stage training process; A compressed feature memory library containing only normal features is constructed, and defect detection is performed on the industrial parts to be inspected based on the compressed feature memory library and the trained defect detection model.

2. The industrial parts defect detection method based on two-stage contrastive learning according to claim 1 is characterized in that: The image dataset of industrial parts collected is specifically: Industrial cameras are used to complete the image data collection of industrial parts, and the collected image data sets are normalized and brightness uniformized.

3. The industrial parts defect detection method based on two-stage contrastive learning according to claim 1 is characterized in that: The abnormal data is generated by performing abnormal data generation on the normal sample images in the training data set to obtain diverse synthetic abnormal samples, specifically: Use a random seed seed to generate a two-dimensional Perlin noise mask P; Using an additional texture image dataset, a random seed is used to select an image as the data source A of the abnormal area; From the training data set T train Randomly select a normal sample image I∈T train , randomly apply a slight rotation less than or equal to the set angle and a rectangular rotation to the normal sample image to obtain an enhanced normal sample image I R ; According to the generated Perlin noise mask, the abnormal data source image is cut and pasted to the normal sample image after data enhancement, thereby obtaining the synthesized abnormal image T G And its corresponding pixel-level label y G .

4. The industrial parts defect detection method based on two-stage contrastive learning according to claim 1 is characterized in that: Through the first stage of training, a discriminant network with coarse-grained anomaly location capability is obtained, specifically: Use the image anomaly generation module G to build a synthetic anomaly dataset T G : Where i = {1,…,N} represents the index of the image in the training set; A pre-trained feature extraction network is used to extract the features of the image. At the same time, a network symmetrical to the feature extraction network is constructed as the discriminant network. When the image enters the feature extraction network and the discriminant network, each layer of the network can obtain the feature map of the image, and the size of each layer of the feature map obtained by the feature extraction network corresponds to the size of each layer of the feature map obtained by the discriminant network. In the discriminant network, the feature map of each layer is first upsampled, and then the feature fusion is completed through jump connection. After being spliced ​​with the feature map of the corresponding size in the feature extraction network, the local context information of the feature map is obtained after convolution, thereby capturing the details and semantic information of the image on the features of different layers. During the training process, the weights of the feature extraction network are frozen, and only synthetic abnormal images are used to train the discriminant network. Using synthetic abnormal image I G and the corresponding pixel-level label y G To train the discriminant network to obtain the coarse-grained anomaly location capability; when the synthetic abnormal image I G After being input into the network, a predicted feature map is output after passing through the feature extraction network and the discriminant network. Use the output and label to first calculate the loss function to update the learnable parameters of the discriminant network; in order to alleviate the imbalance between normal samples and abnormal samples, use focal loss as the first loss function L for discriminant network training dis , specifically: Among them, α t is a scaling factor associated with category t, γ is an adjustable parameter, and p t Corresponding to the predicted pixel classification, when p t =1 indicates abnormal category, p t =0 indicates the normal category.

5. The industrial parts defect detection method based on two-stage contrastive learning according to claim 1 is characterized in that: The data after the first stage training is used for the second stage training, and the contrastive learning framework is used to complete the adaptive fine-tuning of the feature extraction network, and learn the similarities between normal samples and the differences between normal samples and synthetic abnormal samples, specifically: Perform data augmentation on normal samples and obtain two enhanced views v 1 and v 2 ; Construct a contrastive learning network C with two branches, where one branch contains only one projection head, and the other branch contains one projection head and one prediction head, and the two branches share a feature extraction network; The two enhanced views v 1 and v 2 The two branches are sent to each other respectively, and the features of the last two layers are extracted and then spliced ​​and fused to obtain the projection head output p1 and the prediction head output z2. The one-way cosine similarity loss function value is calculated for the two outputs: Among them, |·|2 represents the Euclidean distance of the output features of the two branches; similarly, the projection head output p2 and the prediction head output z1 are obtained, and the calculation is Calculate the average bidirectional similarity to maintain balance: Among them, the "stop gradient" operation SG is used for z1 and z2 to prevent the collapse problem in the contrastive learning process; when M different data augmentation views are generated for each normal sample, the symmetric similarity loss function is as follows: At the same time, after the synthetic abnormal samples pass through the feature extraction network and the frozen discriminant network, the output is the predicted feature map From this, the focal loss is calculated as the negative sample guided loss function: The overall loss function of the second stage is: L ncl =λ·L sym +(1-λ)·L neg Among them, λ is a hyperparameter that balances the two loss functions.

6. The industrial parts defect detection method based on two-stage contrastive learning according to claim 5 is characterized in that: The normal samples are enhanced to obtain two enhanced views v 1 and v 2 , specifically: After the positive sample enhancement module S, two different data enhancement views v are generated 1 and v 2 , in the positive sample enhancement module S In the algorithm, the image is first cropped at random positions to obtain image blocks at different positions of the image; secondly, Gaussian filtering is performed on these image blocks to make specific features present in different states; finally, these image blocks are randomly flipped horizontally to reduce the model's dependence on the image direction and alleviate directional bias.

7. The industrial parts defect detection method based on two-stage contrastive learning according to claim 1 is characterized in that: The construction of a compressed feature memory library containing only normal features is specifically as follows: In the inference phase, the fine-tuned feature extraction network F is used as the backbone network, and the input is from the test dataset T test The test image I t ∈T test , extract adaptive features from the second and third layers, and combine them to obtain the fusion features of the image, thereby obtaining both shallow texture information and deep semantic content; then, these fusion features are used to construct the original memory library M O ; A greedy core set sampling algorithm is used to identify the most representative features for constructing the compressed feature memory library M. * ; Get the compressed feature memory library M * After that, the feature subset c closest to the image feature is sampled. * , calculate the maximum Euclidean distance between the image feature and the nearest feature in the feature subset to obtain the pixel-level anomaly score s of the test image t ; Since the pixel-level anomaly score is a feature map, the maximum value of the pixel-level anomaly scores is used as the image-level anomaly score.

8. An industrial parts defect detection system based on two-stage contrastive learning, characterized in that: An industrial parts defect detection method based on two-stage contrastive learning applied to any one of claims 1-7, comprising an image acquisition module, an image anomaly generation module, a discriminant network pre-training module, a negative guided contrastive learning module, and a compressed feature memory library construction module; The image acquisition module is used to acquire an image data set of industrial parts; the image data set includes a training data set and a test data set; The image anomaly generation module is used to generate abnormal data from normal sample images in the training data set to obtain diverse synthetic abnormal samples; The discriminant network pre-training module is used to input the normal samples and synthetic abnormal samples into a pre-established defect detection model for two-stage training, and obtain a discriminant network with coarse-grained abnormality positioning capability through the first-stage training. The discriminant network is used to assist the training process of the feature extraction network; The negative guided contrastive learning module is used to perform the second stage training on the data after the first stage training, and use the contrastive learning framework to complete the adaptive fine-tuning of the feature extraction network, and learn the similarities between normal samples and the differences between normal samples and synthetic abnormal samples; The compressed feature memory library construction module is used to construct a compressed feature memory library containing only normal features, and perform defect detection on the industrial parts to be inspected based on the compressed feature memory library and the trained defect detection model.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores computer program instructions that can be executed by the at least one processor, and the computer program instructions are executed by the at least one processor so that the at least one processor can execute the industrial parts defect detection method based on two-stage comparative learning as described in any one of claims 1-7.

10. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the industrial parts defect detection method based on two-stage contrast learning described in any one of claims 1 to 7 is implemented.

Images