Abnormal perception-based small sample defect detection method and system

Abstract

The invention discloses a small sample defect detection method and system based on abnormal perception. According to the system, an unlabeled normal set and a labeled training set are adopted to carry out multiple times of iterative training to convergence through a target detection training model with an anomaly perception module so as to obtain a basic model, and detection of a defect target and positioning of unknown anomaly are realized through a reasoning module. And the detected unknown defect data is imported into a labeling module to realize labeling updating of the unknown defect data, and a training data set is iteratively improved. According to the method, the detection capability of the small sample defect detection model on unknown defects is improved, and particularly, the unknown defects are positioned through the anomaly sensing module, so that the defect detection rate is increased, and upgrading iteration of the model is also facilitated.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, in particular, to the field of defect detection in computer vision, and more particularly, to a small sample defect detection method and system based on abnormality perception. Background Art

[0002] Visual defect detection is an important field of artificial intelligence (AI) in industrial scenarios. It uses industrial cameras to collect images of objects to be inspected and determines the location and information of defects by analyzing image features. Visual defect detection tasks in industrial scenarios usually use anomaly detection and target detection methods.

[0003] Among them, the anomaly detection task is an unsupervised detection method under a large number of positive samples. The model can achieve a good detection effect by simply providing defect-free positive sample data. However, in practical applications, when the foreground (defective area) and background (non-defective area) of the image are complex and the distinction is not high, it is very easy to cause false detection, and the model performance will be greatly affected.

[0004] The target detection task is to train a model with high accuracy on a dataset with a large amount of labeled resources to perform target recognition. However, in the actual production environment, there are problems such as the difficulty in providing abnormal samples, the large number of unknown type defects, and the high cost of manual labeling. As a result, the trained model lacks direct enhancement of the features of small samples of new classes, resulting in poor generalization detection capabilities of the model obtained by this method for new classes.

[0005] In summary, although the existing technologies can already achieve high-precision defect detection tasks, there are still problems with small sample detection. Especially in actual scenarios, due to various factors such as process, environment, and equipment, defect data presents a situation of many types, few samples, and difficulty in collection. The existing defect classification often cannot cover the defect generation mechanism of multi-scale unknown external factors in the real production environment, resulting in the problem that the trained detection model cannot locate and identify unknown but obvious defects. At the same time, since they cannot be detected, it is impossible to automatically collect such small sample defects during the detection process to supplement the training data set for iterative upgrades. Summary of the invention

[0006] Purpose of the invention: In order to solve the problems existing in the prior art, the present invention provides a small sample defect detection method and system based on abnormality perception.

[0007] To achieve the above object, the present invention can adopt the following technical solutions: According to the first aspect of the present invention, a defect detection method based on abnormality perception is designed, and the method includes: S1, obtaining a normal set and a training set, and using the training set to train the target detection basic model to obtain the target detection basic model, using the normal set for pre-training to extract features to the normal space, and using the abnormality perception module to obtain defective targets and target areas. Among them, the normal set contains defect-free sample images; the training set includes multiple defect classifications, each classification has multiple samples with classification labels, and the classification labels include classification category labels and target detection frame position labels; S2, using the normal set and the training set to iterate the training model composed of the target detection basic model, the abnormality perception module, the multi-scale feature extraction, the spatial attention module, the residual network module, and the perception loss module for multiple times until convergence to obtain the final small sample defect detection model.

[0008] In some embodiments of the present invention, each iteration training of the present invention includes: S21, extracting the original feature area of ​​each sample in the normal set to the normal space using an unsupervised anomaly detection method; S22, extracting the original feature area of ​​each sample in the training set using a target detection basic model; S23, extracting the original features of the adjusted defect area of ​​each sample in the training set adjusted by step S22 using a multi-scale feature extraction network; S24, using a spatial attention module to perform enhancement processing based on the original features of each sample in the training set to obtain the enhanced features of each sample in the training set; S25, using a target detection module to extract the original features of each sample in the training set based on all samples in the training set. The target detection box position label and predicted target detection box position are used to calculate the target detection box position regression loss; S26, the abnormal perception module is used to adjust the normal space in the normal set extracted based on steps S21, S24, and S25 and the target area and regression loss of each sample in the training set; S27, the residual network module is used to obtain the detection box position and abnormal feature weight of the target area based on the feature normal space in the normal set and the defect feature in the training set; S28, through the perception loss module, based on S26 and S27, it is determined whether it is an unknown abnormality and the target detection box position regression loss and feature weight are calculated and the parameters of the training model are updated.

[0009] In some embodiments of the present invention, the training model also includes a perception loss module. Each iterative training of the present invention also includes: S28', using the loss value and feature weight obtained by the abnormal perception module, and based on the loss value obtained by the target detection model network, determining whether the defect type at the target position is a known defect or an unknown defect; S29' using the abnormal perception loss calculation formula to obtain the target detection box position and regression loss to update the parameters of the training model.

[0010] In some embodiments of the present invention, the pre-trained target detection basic model is any one of the following networks: YOLOv5, YOLOX, YOLOv7.

[0011] According to a second aspect of the present invention, a defect detection system is provided, which comprises: S31, acquiring an input image; S32, performing defect detection on the input image obtained in step S31 using a small sample defect detection model obtained by the method according to the first aspect of the present invention to obtain defect positions and target defect classifications in the image; S33, providing storage, display, correction and export functions for the defect positions and classifications obtained in step S32 to supplement existing training data sets and realize iterative upgrades of detection models.

[0012] Compared with the prior art, the advantages of the present invention are as follows: the present invention aims at the problem that unknown defects cannot be detected, by introducing a positive sample data set, and based on the classification target features of all samples in the training set, a more focused defect feature map is formed through multi-scale feature recognition and spatial attention mechanism, thereby improving the positioning capability of defect targets; at the same time, an abnormal perception module is introduced, and a normal space is constructed by using feature extraction and feature mapping methods, thereby realizing the discrimination of abnormal areas; combining target detection and normal space judgment of targets and anomalies, through the calculation of the perception loss module, the final small sample defect detection model can have the ability to perceive and locate unknown anomalies while having the ability to recognize known defect categories, thereby solving the problem that the existing model cannot detect and collect unknown defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a schematic diagram of the method structure.

[0014] Figure 2 It is a flowchart of the training process of the abnormal perception method.

[0015] Figure 3 It is a schematic diagram of the perceptual loss module.

[0016] Figure 4 It is a schematic diagram of a small sample defect detection system. Implementation

[0017] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but do not limit the present invention in any form. It should be pointed out that, for those of ordinary skill in the art, several variations and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0018] As mentioned in the background technology section, the existing technology only trains the target detection model at the level of known classification results. However, in actual scenarios, due to various factors such as process, environment, and equipment, defect data is diverse, with few samples and difficult to collect. This leads to the problem that the trained target detection model cannot locate and identify unknown but obvious defects.

[0019] In order to solve the above problems, the solution provided by the present invention is to introduce the concept of normal space in addition to the basic target detection task in the process of training the target detection model to obtain a small sample target detection model, and adopt an abnormal perception mechanism to improve the detection ability of unknown small sample defects, so that it can detect and locate unknown small samples while facilitating the iterative training of subsequent target detection models.

[0020] In summary, the present invention first provides a small sample defect detection method and system based on abnormality perception, such as Figure 1 As shown, the method includes: S1, obtaining a normal set and a training set, and using the training set to train the target detection basic model to obtain the target detection basic model, using the normal set to pre-train the normal space for feature extraction, and using the abnormal perception module to obtain the defect target and target area. Among them, the normal set contains defect-free sample images; the training set includes multiple defect classifications, each classification has multiple samples with classification labels, and the classification labels include classification category labels and target detection frame position labels.

[0021] According to one embodiment of the present invention, step S2 includes: S2, using a normal set and a training set to iterate the training model consisting of the target detection basic model, the abnormal perception module, the feature extraction module, the feature mapping module, the multi-scale feature extraction, the spatial attention module, the residual network module, and the perception loss module for multiple times until convergence to obtain the final small sample defect detection model.

[0022] Among them, the present invention obtains the overall image information of normal samples in the form of feature vectors of corresponding samples by introducing a feature extraction module to form the original image features of each normal set sample. The feature mapping module introduced later will transform the original image features of the samples in the feature space and process the sample features with the new data distribution. The purpose of doing so is to eliminate the distribution bias carried by the original image features to help the model achieve the expected ability to perceive abnormalities. Otherwise, when the image features carrying data distribution bias are input into the network for subsequent processing, the perception ability of abnormal features and their morphological structures that should be possessed will be affected by the bias-oriented influence, thereby ultimately causing the model to misdetect or miss detection.

[0023] In addition, in order to further improve the perception and recognition ability of the final unknown anomaly, the present invention also introduces an abnormal perception module in the training model, constructs a normal space through the module, and in the process of converting the image feature information in the feature space, compresses the image features through the memory bank and helps the model to remember the common features in the normal set. The memory bank in the initial state is composed of the initial samples randomly selected from the normal set after normalization. Each subsequent training state will be continuously iterated and updated in this process, and it will not change during the test phase. After completing the training phase, the memory bank will save the homogeneous features of the images in the memory normal set, which retains the typical feature expressions contained in the normal set. Similar to the feature reconstruction method, both compare the image features, but this method is in a high-dimensional feature space. At the same time, due to the introduction of the current module, the mapping method is improved, and the association between each sample in the feature space is strengthened.

[0024] Finally, the present invention also designs a perceptual loss module, which calculates the degree of abnormality of pixels or regions in the normal set and the normal space and target feature space in the training set based on the residual network, and uses Gaussian filtering to parameterize the abnormal score map to obtain a smooth boundary. Since the abnormal score map is the feature difference result quantified by the model, there is always a local maximum value. Therefore, the maximum value of the abnormal score map is taken as the abnormal detection score S of each image, that is, the abnormal feature weight. After obtaining the abnormal score distribution map, the abnormal score is thresholded to identify which pixels or regions M are considered to be abnormal. At the same time, the target area obtained by training the target detection basic model is defined as N, and the loss is loss. d ,The abnormal perception loss calculation formula is used to calculate the target detection box position regression loss and feature weights and update the parameters of the training model.

[0025] The calculation formula of abnormal perception loss P-Loss is defined as: P-Loss = IOU(M, N)* loss d + S When the abnormal area and the target area do not overlap, the abnormal area is defined as an unknown area, and the unknown label and regression loss are assigned to update the training parameters of the target detection basic model.

[0026] According to one embodiment of the present invention, the target detection basic model in the present invention adopts the YOLO series model. According to one embodiment of the present invention, the target detection basic model is any one of the following models: YOLOv5, YOLOX, YOLOv7. These models can realize basic target detection and target area adjustment. For the convenience of understanding, the part used to realize the target detection function is referred to as the target detection basic model in the present invention, and the part used to realize the target area adjustment is referred to as the abnormal perception module. After the target detection basic model is trained with the training set, the detection target, target area and loss function can be obtained. During the training process, the present invention uses the abnormal perception module to evaluate and adjust the normal space obtained by the positive sample training and the feature area information in the target detection basic model training, and iteratively update the detection target, target area and loss function of the target detection basic model. Since the network structure and functions used in the training model are basically known to those skilled in the art, the present invention does not elaborate on the specific functions of each submodule and subnetwork in the training model, and only introduces the present invention from the level of the training method of the small sample abnormal perception model.

[0027] In order to better understand the present invention, the present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0028] Figure 2 The figure shows a training process diagram of a small sample defect detection model based on abnormal perception. The training model includes a target detection basic model, an abnormal perception module, a feature extraction module, a feature mapping module, a multi-scale feature extraction, a spatial attention module, a residual network module, and a perception loss module. The arrows between modules represent the specific data information transmitted between modules. In the process of training the constructed training model using the normal set and the training set, the normal set and the training set are used as the input of the model for multiple iterative training until convergence, wherein the normal set contains defect-free sample images; the training set includes multiple defect classifications, each classification has multiple samples with classification labels, and the classification labels include classification category labels and target detection frame position labels.

[0029] According to one embodiment of the present invention, each iterative training in the present invention includes: S21, extracting the original feature area of ​​each sample in the normal set to the normal space using an unsupervised anomaly detection method; S22, extracting the original feature area of ​​each sample in the training set using a target detection basic model; S23, extracting the original features of the adjusted defect area of ​​each sample in the training set adjusted by step S22 using a multi-scale feature extraction network; S24, using a spatial attention module to perform enhancement processing based on the original features of each sample in the training set to obtain the enhanced features of each sample in the training set; S25, using a target detection module to extract the original features of each sample in the training set based on all the samples in the training set. The target detection box position label of the sample and the predicted target detection box position calculate the target detection box position regression loss; S26, use the abnormal perception module to adjust the normal space in the normal set extracted based on steps S21, S24, and S25 and the target area and regression loss of each sample in the training set; S27, use the residual network module based on the feature normal space in the normal set and the defect feature in the training set to obtain the detection box position and feature weight of the target area; S28, use the perception loss module to determine whether it is an unknown anomaly based on S26 and S27, calculate the target detection box position regression loss and feature weight, and update the parameters of the training model.

[0030] In step S21, the feature extraction module and the feature mapping module are used to extract the image features in the normal set to form a normal space. The normal space uses a memory bank to perform information compression operations in an unsupervised environment, which is particularly effective in the abnormal perception module. By compressing and storing homogeneous features, the effect of filtering the background complexity of industrial products can be achieved. In order to work with the normal space, an adapter is designed that does not destroy the sample distribution of the source feature space, maintains the integrity of its feature information, and can be losslessly compressed by the memory bank as much as possible, namely, a feature descriptor. It consists of a simple 1x1 convolution block that rearranges the pre-trained feature map. At the same time, on the basis of retaining the original feature information, position encoding is added to increase the spatial perception ability, which is called Position Encoder. Position encoding is very important for ensuring the position invariance of some categories of images. Among them, the position information calculated by the formula is added according to the axis position in the height and width dimensions of the image feature information, and the channel position of the feature map is used as a condition, and estimated by conditional normalization.

[0031] Since the steps described from step S22 to step S25 are methods known to those skilled in the art, the present invention will not be described in detail. The other steps are described in detail below for a better understanding of the present invention.

[0032] like Figure 3As shown, in steps S26 to 28: S28', the target area and feature weights obtained by the abnormal perception module are used, and based on the loss function of the target detection model network, it is determined whether the defect type at the target position is a known defect or an unknown defect; S29' adopts the abnormal perception loss calculation formula to obtain the target detection box position regression loss to update the parameters of the training model.

[0033] Figure 4 The figure shows a schematic diagram of a small sample defect detection system based on abnormal perception. The system includes image acquisition, model training, model deployment, model reasoning and annotation modules. The arrows between modules represent the specific data information transmitted between modules. S31. Using the normal set and the training set as the data source, the model training module, i.e., the model training method based on abnormal perception, is used to perform multiple rounds of iterative training to obtain the model; S32. The model is reasoned and deployed through the model deployment module in the system; S33. When the system is running, the image to be detected is obtained through image acquisition; S34. The collected image is input into the model reasoning module, and the detection result is output; S35. All known / unknown defects are saved in the database, and subsequent operations are performed according to the business logic; S36. Steps S31 to S35 are repeated to obtain a large amount of detected defect data; S37. The detected defect data set is judged, and the unknown / known defects are displayed through the annotation module, and the defect area and confidence are displayed. Through the secondary judgment of the defect, the classification label annotation is adjusted; S38. The newly added data is added to the training set, and the above steps are repeated to realize the iterative update of the model.

[0034] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

[0035] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0036] The present invention may be a system, method and / or computer program product, and its essence or the part that contributes to the relevant technology may be embodied in the form of a software product. The computer software product may be stored in a computer-readable storage medium, and includes a number of instructions to use at least one computer device to execute the methods described in various embodiments or certain parts of the embodiments.

Claims

1. A small sample defect detection method and system based on abnormality perception, characterized in that: The method includes: S1, obtaining a normal set and a training set, and using the training set to train the target detection basic model to obtain the target detection basic model, using the normal set for pre-training to extract features to the normal space, and using the abnormal perception module to obtain defective targets and target areas. Wherein, the normal set contains defect-free sample images; the training set includes multiple defect classifications, each classification has multiple samples with classification labels, and the classification labels include classification category labels and target detection frame position labels; S2, using the normal set and the training set to iterate the training model composed of the target detection basic model, the abnormal perception module, the multi-scale feature extraction, the spatial attention module, the residual network module, and the perception loss module for multiple times until convergence to obtain the final small sample defect detection model.

2. The method according to claim 1, characterized in that Each iteration training includes: S21, using the unsupervised anomaly detection method to extract the original feature area of ​​each sample in the normal set to the normal space; S22, using the target detection basic model to extract the original feature area of ​​each sample in the training set respectively; S23, using the multi-scale feature extraction network to extract the original features of the adjusted defect area of ​​each sample in the training set adjusted by step S22; S24, using the spatial attention module to enhance the original features of each sample in the training set to obtain the enhanced features of each sample in the training set; S25, using the target detection module to enhance the target detection box position labels and Predict the target detection box position and calculate the target detection box position regression loss; S26, use the abnormal perception module to adjust the normal space in the normal set extracted based on steps S21, S24, and S25 and the target area and regression loss of each sample in the training set; S27, use the residual network module to obtain the detection box position and abnormal feature weight of the target area based on the feature normal space in the normal set and the defect feature in the training set; S28, determine whether it is an unknown abnormality based on S26 and S27 through the perception loss module, calculate the target detection box position regression loss and feature weight, and update the parameters of the training model.

3. The method according to claim 2, characterized in that The pre-trained target detection basic model is any one of the following networks: YOLOv5, YOLOX, YOLOv7.

4. The method according to claim 2, characterized in that: The anomaly perception module is introduced, and a memory bank is used to store the features corresponding to each sample selected from the normal set. The features are derived from the basic feature extraction model and are different from the original features of the sample. The screening process is random at the beginning of training, so the stored features are also random. During the training process, the selected sample features continuously update the stored features, and the stored features will achieve the target feature effect, which is similar to the anomaly perception function.

5. According to the method of claim 4, the abnormal perception module further comprises a perception loss module, and each iterative training of the present invention further comprises: S28', using the loss value and feature weight obtained by the abnormal perception module, and based on the loss value obtained by the target detection model network, determining whether the defect type at the target location is a known defect or an unknown defect; S29' uses the abnormal perception loss calculation formula to obtain the target detection box position and regression loss to update the parameters of the training model.

6. The method according to claim 5, characterized in that The abnormal perception loss formula P-Loss is calculated using the following formula: P-Loss = IOU(M, N)* loss d + S, where S is the abnormal feature weight, M is the abnormal area, N is the target detection area, and loss d Target regression loss for object detection.

7. A target detection method, characterized in that: The detection method includes: S31, obtaining an input image; S32, using a small sample defect detection model obtained according to any one of the methods described in claims 1 to 6 to perform defect detection on the input image obtained in step S31 to obtain the defect position and defect classification in the image; S33, providing storage, display, correction and export functions for the defect position and classification obtained in step S32 to supplement the existing training data set and realize iterative upgrading of the detection model.

8. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of any method described in claims 1 to 6 and 7.

9. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs, which, when executed by the one or more processors, enables the electronic device to implement the steps of the method as claimed in any one of claims 1 to 6 and 7.

Images