A defect detection method and related device for industrial scenarios

Abstract

In order to solve the technical problem that defect detection networks are prone to over-detection and missed detection in current reconstruction-based defect detection methods, the present application provides a defect detection method and related devices for industrial scenarios, extracts high semantic features and low semantic features from an image to be identified, and after adapting the two sets of key features, inputs the adapted features into a defect detection model to obtain defect detection results, wherein the defect detection model includes a cross-feature reconstruction network and a defect discrimination network, the cross-feature reconstruction network is used to cross-reconstruct the adapted high semantic features and the adapted low semantic features of defects, respectively, to obtain the matched high semantic features and the adapted low semantic features of non-defects, and calculate the defect location information, the defect discrimination network is used to use the defect location information as input, perform defect scoring on each pixel in the input features, and output accurate defect detection results, which can reduce the missed detection rate and / or over-detection rate in the product defect detection process.

Description

Technical Field

[0001] The present application relates to a defect detection method, and specifically to a defect detection method and related devices for industrial scenarios. Background Art

[0002] Defect detection has always been an important research topic in the field of industrial production and has been widely used in various industrial scenarios, such as 3C, machinery manufacturing, automobiles, home appliances, aerospace, etc. With the advancement of technology, manual inspection has gradually been replaced by automated inspection methods based on deep learning.

[0003] Defect detection methods based on supervised learning require a large number of defect samples to train the model, but there are problems such as difficulty in obtaining a large number of training samples and the need to predict the defect type. In recent years, the problem of unsupervised defect detection has received widespread attention, and the defect detection method based on reconstruction is currently the most widely used method. This method locates anomalies by analyzing the reconstruction error, and usually only requires normal samples for training. However, defect detection networks based on reconstruction methods usually have two common problems: (1) The normal area is sometimes not completely consistent with the original image, resulting in a large reconstruction error in the normal area, causing the model to mistake the normal area for a defect, resulting in over-inspection; (2) The defect area is sometimes unable to be completely repaired to normal, resulting in a small reconstruction error in the defect area, causing the model to easily mistake the defect area for a normal area, resulting in missed detection. All of the above problems will directly affect the detection accuracy. Summary of the invention

[0004] The present application aims to solve the technical problem that the defect detection network is prone to over-detection and under-detection in the current reconstruction-based defect detection method, and provides a defect detection method and related devices for industrial scenarios.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In the first aspect, the present application proposes a defect detection method for industrial scenarios, including:

[0007] Obtain an image to be recognized;

[0008] Extract features layer by layer from the image to be identified, and obtain high semantic features and low semantic features as two sets of key features;

[0009] Adapting the high semantic feature and the low semantic feature respectively to obtain an adapted high semantic feature and an adapted low semantic feature, which are respectively recorded as an adapted high semantic feature and an adapted low semantic feature;

[0010] Inputting the adapted high semantic features and the adapted low semantic features into a defect detection model to obtain a defect detection result;

[0011] The defect detection model includes a cross-feature reconstruction network and a defect discrimination network;

[0012] The cross-feature reconstruction network is used to repair and reconstruct defective adaptive high semantic features into non-defective low semantic adaptive features to obtain reconstructed low semantic adaptive features, and repair and reconstruct defective adaptive low semantic features into non-defective high semantic adaptive features to obtain reconstructed high semantic adaptive features; fuse high semantic defect position information and low semantic defect position information to obtain fused defect position information; wherein the high semantic defect position information is the Euclidean distance between the defective adaptive high semantic features and the reconstructed high semantic adaptive features, and the low semantic defect position information is the Euclidean distance between the defective adaptive low semantic features and the reconstructed low semantic adaptive features;

[0013] The defect discrimination network is used to take the fused defect position information as input, perform defect scoring on each pixel in the fused defect position information, and output a defect detection result.

[0014] Furthermore, the training method of the cross-feature reconstruction network includes:

[0015] Extract features of the training sample images layer by layer to obtain high semantic features and low semantic features of the training samples;

[0016] Adapting the high semantic features of the training samples and the low semantic features of the training samples to obtain adapted high semantic features of the training samples and adapted low semantic features of the training samples;

[0017] Add feature-level mask noise to all channels of the adapted training sample high semantic features and the adapted training sample low semantic features, respectively, to obtain high semantic simulated abnormal features and low semantic simulated abnormal features;

[0018] The high-semantic simulated abnormal features and the low-semantic simulated abnormal features are input into a cross-feature reconstruction network for training to obtain a trained cross-feature reconstruction network.

[0019] Further, the step of adding feature-level mask noise to all channels in the adapted training sample high semantic features and the adapted training sample low semantic features to obtain high semantic simulated abnormal features and low semantic simulated abnormal features includes:

[0020] Generate a binary mask image through Perlin noise;

[0021] Multiplying the binary mask image by Gaussian noise pixel by pixel to obtain feature-level mask noise;

[0022] The adapted high semantic features of the training samples and the adapted low semantic features of the training samples are respectively added pixel by pixel to the feature-level mask noise to obtain high semantic simulated abnormal features and low semantic simulated abnormal features.

[0023] Furthermore, the training method of the defect discrimination network includes:

[0024] Obtaining the real simulated abnormal position information and the simulated abnormal position information predicted after reconstruction by the cross-feature reconstruction network;

[0025] The real simulated abnormal position information and the simulated abnormal position information predicted after reconstruction by the cross-feature reconstruction network are input into the defect discrimination network for training to obtain the trained defect discrimination network.

[0026] Furthermore, the step of extracting features layer by layer from the image to be identified includes:

[0027] High semantic features and low semantic features are extracted from the image to be recognized through a large visual model.

[0028] Furthermore, the adapting the high semantic features and the low semantic features includes:

[0029] A single-layer fully connected network structure is used to adapt the high semantic features and the low semantic features.

[0030] Furthermore, the cross-feature reconstruction network adopts an encoder-decoder structure of the Mamba architecture.

[0031] In the second aspect, the present application proposes a defect detection system for industrial scenarios, including:

[0032] An acquisition module, used for acquiring an image to be recognized;

[0033] An extraction module is used to extract features from the image to be identified layer by layer, and obtain high semantic features and low semantic features as two groups of key features;

[0034] An adaptation module, used for adapting the high semantic feature and the low semantic feature respectively, to obtain an adapted high semantic feature and an adapted low semantic feature, which are respectively recorded as an adapted high semantic feature and an adapted low semantic feature;

[0035] A detection module, used for inputting the adapted high semantic features and the adapted low semantic features into a defect detection model to obtain a defect detection result;

[0036] The defect detection model includes a cross-feature reconstruction network and a defect discrimination network;

[0037] The cross-feature reconstruction network is used to repair and reconstruct defective adaptive high semantic features into non-defective low semantic adaptive features to obtain reconstructed low semantic adaptive features, and repair and reconstruct defective adaptive low semantic features into non-defective high semantic adaptive features to obtain reconstructed high semantic adaptive features; fuse high semantic defect position information and low semantic defect position information to obtain fused defect position information; wherein the high semantic defect position information is the Euclidean distance between the defective adaptive high semantic features and the reconstructed high semantic adaptive features, and the low semantic defect position information is the Euclidean distance between the defective adaptive low semantic features and the reconstructed low semantic adaptive features;

[0038] The defect discrimination network is used to take the fused defect position information as input, perform defect scoring on each pixel in the fused defect position information, and output a defect detection result.

[0039] In a third aspect, the present application proposes an electronic device, comprising: a memory and one or more processors; the memory is coupled to the processor; wherein computer program code is stored in the memory, and the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the steps of the above-mentioned defect detection method for industrial scenarios.

[0040] In a fourth aspect, the present application proposes a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the defect detection method for industrial scenarios as described above are implemented.

[0041] Compared with the prior art, this application has the following beneficial effects:

[0042] The present application proposes a defect detection method for industrial scenarios, which extracts key features from the image to be identified, which are high semantic features and low semantic features, respectively. After adapting the two sets of key features, the adapted features are input into the defect detection model to obtain defect detection results, wherein the defect detection model includes a cross-feature reconstruction network and a defect discrimination network. The cross-feature reconstruction network is used to repair and reconstruct the defective adapted features into non-defective adapted features, and calculate the defect location information. The defect discrimination network is used to use the defect location information as input, perform defect scoring on each pixel in the input feature, and output accurate defect detection results. The present application repairs and reconstructs the defective adapted features with the help of the feature reconstruction network, and then detects them through the defect discrimination network. The defect detection model constructed by the construction method is used to perform product defect detection, which can reduce the missed detection rate and / or over-detection rate in the product defect detection process, effectively improve the accuracy of defect detection, and ensure the efficiency of defect detection.

[0043] The present application also proposes a defect detection system for industrial scenarios, an electronic device and a computer storage medium, which have all the advantages of the above-mentioned defect detection method for industrial scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0045] Figure 1 This is a first flow chart of a defect detection method for industrial scenarios in this application.

[0046] Figure 2 This is a second flow chart of a defect detection method for industrial scenarios in this application.

[0047] Figure 3 Schematic diagram of the pre-training process of the cross-feature reconstruction network in an embodiment of the present application.

[0048] Figure 4 In an embodiment of the present application, a schematic diagram of adding a preset feature perturbation to the feature maps of all channel dimensions in the adapted training sample features.

[0049] Figure 5 This is a schematic diagram of the Mamba architecture in an embodiment of the present application.

[0050] Figure 6 A schematic diagram of a defect detection system for industrial scenarios in this application.

[0051] Figure 7 A schematic diagram of an electronic device according to the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0053] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0054] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0055] In the description of the embodiments of the present application, it should be noted that if the terms "upper", "lower", "horizontal", "inner", etc. appear, the orientation or position relationship indicated is based on the orientation or position relationship shown in the drawings, or the orientation or position relationship in which the invented product is usually placed when used. It is only for the convenience of describing the present application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present application. In addition, the terms "first", "second", etc. are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0056] In addition, if the term "horizontal" appears, it does not mean that the component must be absolutely horizontal, but can be slightly tilted. For example, "horizontal" only means that its direction is more horizontal than "vertical", which does not mean that the structure must be completely horizontal, but can be slightly tilted.

[0057] In the description of the embodiments of the present application, it is also necessary to explain that, unless otherwise clearly specified and limited, the terms "set", "install", "connect", and "connect" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal connection of two components. For ordinary technicians in this field, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0058] Defect detection plays an important role in industrial production. With the rapid development of science and technology, traditional manual inspection methods are gradually being replaced by automated inspection methods based on deep learning. This shift has significantly improved inspection efficiency and accuracy. However, defect detection methods based on supervised learning face many challenges in practical applications, the most prominent of which are the difficulty in obtaining defect samples and the need to predict defect types:

[0059] (1) Limitations of supervised learning defect detection methods

[0060] In industrial production, the collection of defect samples is often subject to many limitations, such as low probability of defect occurrence and diverse defect types, which makes it difficult to obtain a sufficient number of high-quality defect samples for model training. In addition, supervised learning requires that all possible defect types be defined in advance and that samples of these types be included in the training data. However, in actual applications, defect types may be diverse and new defect types may appear at any time, which makes it difficult for supervised learning methods to meet actual needs.

[0061] (2) The rise of unsupervised defect detection methods and reconstruction-based methods

[0062] Given the limitations of supervised learning, unsupervised defect detection methods have received widespread attention in recent years. Among them, reconstruction-based defect detection methods are widely used because they only require normal samples for training. This method locates abnormal areas by analyzing the error between the input image and the reconstructed image. However, the normal area sometimes cannot be completely reconstructed into the original image, resulting in a large reconstruction error, and the model may misjudge the normal area as a defective area. The defective area sometimes cannot be completely repaired to a normal state, resulting in a small reconstruction error, and the model may misjudge the defective area as a normal area.

[0063] Based on the above situation, the present application proposes a defect detection method and related devices for industrial scenarios. The present application is described in detail below in conjunction with embodiments and drawings.

[0064] like Figure 1 As shown, it is a first flow chart of a defect detection method for industrial scenarios of the present application, which may include:

[0065] S101, obtaining an image to be recognized.

[0066] The images to be identified are images that need to be inspected for defects. In practical applications, these images to be identified may come from various sources, such as product photos on industrial production lines, monitoring images of building structures, etc.

[0067] S102, extracting features layer by layer from the image to be recognized, and obtaining high semantic features and low semantic features as two groups of key features.

[0068] In practical applications, image processing techniques such as edge detection, texture analysis, and color space conversion can be used to highlight key information in images. From the input image to be identified, deep learning models are used to automatically learn and extract features of different semantic levels in the image. Generally speaking, low-level features are extracted from low semantic features, such as color, texture, and other information, and high-level features are extracted from high semantic features, such as structure, logic, and other information. Here, two groups of high semantic features and low semantic features of different levels are extracted.

[0069] S103, adapting the high semantic feature and the low semantic feature respectively to obtain an adapted high semantic feature and an adapted low semantic feature, which are recorded as an adapted high semantic feature and an adapted low semantic feature respectively.

[0070] In practical applications, the extracted high-semantic features and low-semantic features can be further processed through adaptation to make them suitable for the input requirements of subsequent defect detection models. This may involve operations such as feature scaling, normalization, and data enhancement to improve the generalization ability and detection accuracy of defect detection models. It is also possible to reduce computational complexity and improve processing efficiency through feature selection or dimensionality reduction.

[0071] S104, inputting the adapted high semantic features and the adapted low semantic features into a defect detection model to obtain a defect detection result.

[0072] Wherein, the defect detection model includes a cross-feature reconstruction network and a defect discrimination network;

[0073] The cross-feature reconstruction network is used to repair and reconstruct defective low-semantic adaptive features into non-defective high-semantic adaptive features, and to repair and reconstruct defective high-semantic adaptive features into non-defective low-semantic adaptive features. The Euclidean distance of the low-semantic adaptive features before and after reconstruction is recorded as low-semantic defect position information, and the Euclidean distance of the high-semantic adaptive features before and after reconstruction is recorded as high-semantic defect position information. The high-semantic defect position information and the low-semantic defect position information are fused to obtain complete defect position information, which is recorded as fused defect position information. In practical applications, deep learning models such as generative adversarial networks or variational autoencoders can be used to repair the features of defective images by learning the feature distribution of a large number of defect-free images.

[0074] The defect discrimination network is used to take the fused defect position information predicted by the cross-feature reconstruction network as input, and to score each pixel in the input feature, and output an accurate defect detection result. In practical applications, the defect discrimination network can use a binary classification or regression network, which can compare the difference between the original adapted features and the reconstructed features, give a probability or score of defect existence for each pixel, and then complete defect detection.

[0075] As an implementation of the above embodiment, the following method may be specifically adopted:

[0076] (1) Obtain the image to be recognized;

[0077] (2) The image to be identified is input into the feature extractor to obtain the high semantic features and low semantic features of the image to be identified. The features are adapted by the feature adapter to obtain the adapted high semantic features and the adapted low semantic features. Then, the features are input into the pre-trained cross-feature reconstruction network to obtain the reconstructed low semantic adapted features and the reconstructed high semantic adapted features. The high semantic defect position information and the low semantic defect position information are calculated by the adapted high semantic features and the adapted low semantic features before and after reconstruction. Among them, the pre-trained cross-feature reconstruction network is obtained by training the initial cross-feature reconstruction network with the training image samples, and is used to repair and reconstruct the high semantic features and the low semantic features of the image to be identified.

[0078] (3) The defect location information predicted by the cross-feature reconstruction network is input into the pre-trained defect discrimination network to perform defect discrimination analysis on the image to be identified. The pre-trained defect discrimination network is obtained by training the initial defect discrimination network using training image samples, and is used to accurately discriminate and analyze the defect location information of the image to be identified.

[0079] like Figure 2 As shown, it is a second flow chart of a defect detection method for industrial scenarios of the present application, which may include:

[0080] S201, obtaining an image to be recognized.

[0081] S202, pre-training the cross-feature reconstruction network.

[0082] The pre-training of the cross-feature reconstruction network can be specifically carried out using the following methods: Figure 3 As shown, this is a schematic diagram of the pre-training process of the cross-feature reconstruction network:

[0083] (1) Obtain training sample images.

[0084] (2) Extract features from the training sample images to obtain high semantic features and low semantic features of the training samples. Feature extraction can be performed using a pre-trained feature extractor with frozen parameters. In practical applications, the feature extractor can use a large visual model, such as DINO (Distillation NO labels, a large visual model) and DINOv2.

[0085] (3) Input the high semantic features of the training samples and the low semantic features of the training samples into the feature adapter for feature adaptation to obtain the adapted training samples, the adapted high semantic features of the training samples and the adapted low semantic features of the training samples. In practical applications, the feature adapter can use a single-layer fully connected network structure.

[0086] (4) Add feature-level mask noise to all channels of the above-mentioned adapted high semantic features of the training samples and the adapted low semantic features of the training samples to obtain high semantic simulated abnormal features and low semantic simulated abnormal features.

[0087] like Figure 4 As shown, it is a schematic diagram of adding preset feature perturbations to the feature maps of all channel dimensions in the adapted high semantic features of the training samples and the adapted low semantic features of the training samples. Preset feature perturbations are added to the feature maps of all channel dimensions in the adapted high semantic features of the training samples and the adapted low semantic features of the training samples. Different from the image-level simulated anomaly, the feature-level mask noise adds constrained perturbations on each channel dimension in the feature. The feature-level mask noise first generates a binary mask map by Perlin noise to constrain the area where the noise is generated, and then multiplies the above binary mask map with Gaussian noise pixel by pixel to obtain feature-level mask noise. The feature-level mask noise is added pixel by pixel to the training sample feature to obtain the high semantic simulated anomaly feature and the low semantic simulated anomaly feature.

[0088] (5) Input the high semantic simulated abnormal features and the low semantic simulated abnormal features into the cross feature reconstruction network for training, and obtain the trained cross feature reconstruction network. The cross feature reconstruction network is trained to be a network that can repair and reconstruct the high semantic features of simulated abnormalities into the low semantic features of normal training samples, and repair and reconstruct the low semantic features of simulated abnormalities into the high semantic features of normal training samples, and calculate the simulated abnormality position information. In practical applications, the feature reconstruction network can adopt the encoder-decoder structure of the Mamba architecture (Mixed-Autodiff Batching for Adam, a feature reconstruction network), such as Figure 5 The figure shows a schematic diagram of the Mamba architecture.

[0089] In practical applications, the cross-feature reconstruction network can adopt a deep learning framework, including an end-to-end open source machine learning platform, such as one or more of Caffe or PyTorch.

[0090] S203, pre-training the feature defect discrimination network.

[0091] The pre-training of the defect discrimination network can be specifically carried out using the following methods:

[0092] (1) Obtaining simulated abnormal features before and after reconstruction of the cross-reconstruction feature network;

[0093] (2) The simulated abnormal position information predicted by the cross-feature reconstruction network and the actual simulated abnormal position information are input into the defect discrimination network for training to obtain a trained defect discrimination network. The defect discrimination network is used to perform defect scoring on each pixel point of the feature map.

[0094] It should be noted that the defect discrimination network can adopt a deep learning framework, including an end-to-end open source machine learning platform, such as one or more defect discrimination networks in Caffe or PyTorch. The network can adopt a single-layer fully connected layer structure, and the ground_truth (reference label) in the training process is an artificially generated known feature-level mask noise.

[0095] S204, fusing the pre-trained cross-feature reconstruction network and the pre-trained defect discrimination network to obtain a defect detection model.

[0096] S205, extracting key high-semantic features and low-semantic features from the image to be recognized by a feature extractor.

[0097] S206, adapting the key features through a feature adapter to obtain adapted high semantic features and adapted low semantic features.

[0098] S207, inputting the image to be identified into a pre-trained defect detection model to determine the fused defect position information corresponding to the image to be identified.

[0099] In practical applications, the pre-trained model can also be tested, which can include:

[0100] (1) Constructing data sets for training, such as mobile phone screen defects, cloth surface defects, etc.

[0101] (2) Annotate the collected images and convert the annotation results into ground_truth in png format;

[0102] (3) Integrate the network in this application into a specific deep learning framework, such as TensorFlow, Caffe, PyTorch, etc.;

[0103] (4) Training the model. According to the category of the object, the image feature reconstruction network and the defect discrimination network are trained in sequence;

[0104] (5) Test model. A data set for testing was constructed and converted into a standard input image of 256×256 size and input into the network to obtain a predicted defect score map. Compared with other methods, this method has a higher defect detection accuracy and can basically meet the requirements of industrial scenarios for defect detection accuracy.

[0105] Here is an example:

[0106] 1. Obtain training sample images.

[0107] 2. Input the training sample image into the parameter-frozen pre-trained visual large model DINOv2 to extract features, and use the features output by the 8th and 10th layers of DINOv2 as the training sample features, where the 8th layer is the low semantic features and the 10th layer is the high semantic features. Input the training sample features into the feature adapter, and output the adapted training sample features, denoted as and .

[0108] 3. Since the tasks and distributions of the pre-training datasets used in industrial inspection images and feature extractors are different, a feature adapter is used to transfer the training image features to the target domain, i.e., the industrial inspection image domain. The feature adapter uses a single-layer fully connected network structure, and adds feature-level mask noise to the adapted training sample features to generate low-semantic simulated abnormal features and high-semantic simulated abnormal features, which are denoted as and .

[0109] The process of generating feature-level mask noise is as follows:

[0110] First, Perlin noise is used to generate a binary mask and the distribution of the noise is determined, which is recorded as , and then use Gaussian noise Pixel by pixel with the above binary mask Multiply, generate , which is the feature-level mask noise, the process can be expressed as:

[0111]

[0112] 4. Simulate abnormal features with high semantics and low semantic simulation abnormal features Input to the cross feature reconstruction network for cross feature repair, and output is the reconstructed high semantic features of the training image And the low semantic features of the reconstructed training image , simulate abnormal features with high semantics Reconstruction repair is the low semantic features of the reconstructed training image , simulate abnormal features with low semantics Reconstruction and restoration are high semantic features of the reconstructed training image The cross-feature reconstruction network adopts an encoder-decoder structure based on the Mamba architecture. The Mamba architecture achieves linear computational complexity while ensuring global perception capabilities. It is suitable for tasks with high computational complexity such as feature reconstruction. The loss function used by the feature reconstruction network is L2 loss, and the reconstruction loss It can be expressed as:

[0113]

[0114] 5. The Euclidean distance between the low semantic features before and after reconstruction is the low semantic simulated abnormal position information, and the Euclidean distance between the high semantic features before and after reconstruction is the high semantic simulated abnormal position information. The high / low semantic simulated abnormal position information is multiplied pixel by pixel to obtain the predicted fused simulated abnormal position information.

[0115] 6. Input the predicted fusion defect location information and the real simulated abnormal location information into the defect discrimination network, which is a pixel-level defect scorer that estimates the normality of each pixel in the feature. The discrimination network expects normal pixels to output positive and defective pixels to output negative. A single-layer fully connected layer is used here. It should be noted that the training process of the defect discrimination network is supervised, and the label is the anomaly mask generated by the known Perlin noise mentioned above. , the loss function of the defect discrimination network The truncated L1 loss and Focal loss are used, which can be expressed as:

[0116]

[0117] The defect location information map output by the defect discrimination network is Gaussian filtered to obtain a smooth boundary and the final defect detection result is obtained. The value of the maximum response pixel in the defect detection result is used as the defect detection score.

[0118] 7. Input the image to be detected into the trained defect detection model. It should be noted that the image may be either a normal image or a defective image, and defect detection and positioning are performed under the premise of being unknown. The image first passes through the feature extractor with the above-mentioned frozen parameters to extract high semantic features and low semantic features, and then the high semantic features and low semantic features are adapted by the feature adapter. During the test, there is no need to go through the step of adding feature-level mask noise mentioned in the training process. The high semantic features and low semantic features output by the adapter are input into the cross-feature reconstruction module for reconstruction, and the fused defect position information is calculated. The fused position information is input into the defect discrimination network, and the feature discrimination network outputs the defect detection result, that is, it determines whether there is a defect in the image to be detected and accurately locates the defect position.

[0119] This application provides a corresponding solution to the problem of industrial defect detection, and in actual scenarios, it can be applied to the surface defect segmentation of objects in most industrial fields, especially for scenarios with high requirements on the accuracy and real-time performance of the segmentation results.

[0120] like Figure 6 As shown, it is a schematic diagram of a defect detection system for industrial scenarios of the present application, which may include:

[0121] An acquisition module, used for acquiring an image to be recognized;

[0122] An extraction module is used to extract features from the image to be identified layer by layer, and obtain high semantic features and low semantic features as two groups of key features;

[0123] An adaptation module, used for adapting the high semantic feature and the low semantic feature respectively, to obtain an adapted high semantic feature and an adapted low semantic feature, which are respectively recorded as an adapted high semantic feature and an adapted low semantic feature;

[0124] A detection module, used for inputting the adapted high semantic features and the adapted low semantic features into a defect detection model to obtain a defect detection result;

[0125] The defect detection model includes a cross-feature reconstruction network and a defect discrimination network;

[0126] The cross-feature reconstruction network is used to repair and reconstruct defective adaptive high semantic features into non-defective low semantic adaptive features to obtain reconstructed low semantic adaptive features, and repair and reconstruct defective adaptive low semantic features into non-defective high semantic adaptive features to obtain reconstructed high semantic adaptive features; fuse high semantic defect position information and low semantic defect position information to obtain fused defect position information; wherein the high semantic defect position information is the Euclidean distance between the defective adaptive high semantic features and the reconstructed high semantic adaptive features, and the low semantic defect position information is the Euclidean distance between the defective adaptive low semantic features and the reconstructed low semantic adaptive features;

[0127] The defect discrimination network is used to take the fused defect position information as input, perform defect scoring on each pixel in the fused defect position information, and output a defect detection result.

[0128] It should be noted that in the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the system embodiments described above are merely schematic. For example, the division of each module is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules can be combined or integrated into another device, or some features can be ignored or not executed. The module described as a separate component may or may not be physically separated. The component displayed as a module may be a physical unit or multiple physical units, that is, it may be located in one place, or it may be distributed in multiple different places. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0129] In addition, each module in each embodiment of the present invention may be integrated into a processing unit, each module may exist physically separately, or two or more modules may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0130] The present application also provides an electronic device, such as Figure 7 As shown, the electronic device may include one or more processors, memories, and communication interfaces.

[0131] The memory, the communication interface and the processor are coupled, for example, the memory, the communication interface and the processor may be coupled together via a bus.

[0132] The communication interface is used for data transmission with other devices. The memory stores computer program code. The computer program code includes computer instructions. When the computer instructions are executed by the processor, the electronic device executes the compilation method of the application in the embodiment of the present application.

[0133] Wherein, the processor can be a processor or a controller, for example, a central processing unit (CPU), a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the present disclosure. The processor can also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of DSP and microprocessors, and the like. The processor can be used to support electronic devices to execute the method steps provided in the above embodiments.

[0134] The bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The above buses can be divided into address buses, data buses, control buses, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0135] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of the defect detection method for industrial scenarios described in any of the above embodiments are implemented.

[0136] The computer-readable storage medium involved in the present application includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or any other form of storage medium known in the technical field.

[0137] For the description of the relevant parts of the defect detection system for industrial scenarios, electronic equipment, and computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the defect detection method for industrial scenarios provided in the embodiments of the present application, which will not be repeated here. In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0138] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A defect detection method for industrial scenarios, characterized in that: include: Obtain an image to be recognized; Extract features layer by layer from the image to be identified, and obtain high semantic features and low semantic features as two sets of key features; Adapting the high semantic feature and the low semantic feature respectively to obtain an adapted high semantic feature and an adapted low semantic feature, which are respectively recorded as an adapted high semantic feature and an adapted low semantic feature; Inputting the adapted high semantic features and the adapted low semantic features into a defect detection model to obtain a defect detection result; The defect detection model includes a cross-feature reconstruction network and a defect discrimination network; The cross-feature reconstruction network is used to repair and reconstruct defective adaptive high semantic features into non-defective low semantic adaptive features to obtain reconstructed low semantic adaptive features, and to repair and reconstruct defective adaptive low semantic features into non-defective high semantic adaptive features to obtain reconstructed high semantic adaptive features; The high semantic defect position information and the low semantic defect position information are fused to obtain fused defect position information; wherein the high semantic defect position information is the Euclidean distance between the defective adapted high semantic feature and the reconstructed high semantic adapted feature, and the low semantic defect position information is the Euclidean distance between the defective adapted low semantic feature and the reconstructed low semantic adapted feature; The defect discrimination network is used to take the fused defect position information as input, perform defect scoring on each pixel in the fused defect position information, and output a defect detection result.

2. According to claim 1, a defect detection method for industrial scenarios is characterized in that: The training method of the cross-feature reconstruction network includes: Extract features of the training sample images layer by layer to obtain high semantic features and low semantic features of the training samples; Adapting the high semantic features of the training samples and the low semantic features of the training samples to obtain adapted high semantic features of the training samples and adapted low semantic features of the training samples; Add feature-level mask noise to all channels of the adapted training sample high semantic features and the adapted training sample low semantic features, respectively, to obtain high semantic simulated abnormal features and low semantic simulated abnormal features; The high-semantic simulated abnormal features and the low-semantic simulated abnormal features are input into a cross-feature reconstruction network for training to obtain a trained cross-feature reconstruction network.

3. A defect detection method for industrial scenarios according to claim 2, characterized in that: The step of adding feature-level mask noise to all channels in the adapted high semantic features of the training samples and the adapted low semantic features of the training samples to obtain high semantic simulated abnormal features and low semantic simulated abnormal features comprises: Generate a binary mask image through Perlin noise; Multiplying the binary mask image by Gaussian noise pixel by pixel to obtain feature-level mask noise; The adapted high semantic features of the training samples and the adapted low semantic features of the training samples are respectively added pixel by pixel to the feature-level mask noise to obtain high semantic simulated abnormal features and low semantic simulated abnormal features.

4. The defect detection method for industrial scenarios according to claim 3 is characterized in that: The training method of the defect discrimination network comprises: Obtaining the real simulated abnormal position information and the simulated abnormal position information predicted after reconstruction by the cross-feature reconstruction network; The real simulated abnormal position information and the simulated abnormal position information predicted after reconstruction by the cross-feature reconstruction network are input into the defect discrimination network for training to obtain the trained defect discrimination network.

5. A defect detection method for industrial scenarios according to claim 4, characterized in that: Extracting features layer by layer from the image to be identified includes: High semantic features and low semantic features are extracted from the image to be recognized through a large visual model.

6. A defect detection method for industrial scenarios according to claim 5, characterized in that: The adapting the high semantic feature and the low semantic feature includes: A single-layer fully connected network structure is used to adapt the high semantic features and the low semantic features.

7. The defect detection method for industrial scenarios according to claim 6 is characterized in that: The cross-feature reconstruction network adopts the encoder-decoder structure of the Mamba architecture.

8. A defect detection system for industrial scenarios, characterized in that: include: An acquisition module, used for acquiring an image to be recognized; An extraction module is used to extract features from the image to be identified layer by layer, and obtain high semantic features and low semantic features as two groups of key features; An adaptation module, used for adapting the high semantic feature and the low semantic feature respectively, to obtain an adapted high semantic feature and an adapted low semantic feature, which are respectively recorded as an adapted high semantic feature and an adapted low semantic feature; A detection module, used for inputting the adapted high semantic features and the adapted low semantic features into a defect detection model to obtain a defect detection result; The defect detection model includes a cross-feature reconstruction network and a defect discrimination network; The cross-feature reconstruction network is used to repair and reconstruct defective adaptive high semantic features into non-defective low semantic adaptive features to obtain reconstructed low semantic adaptive features, and to repair and reconstruct defective adaptive low semantic features into non-defective high semantic adaptive features to obtain reconstructed high semantic adaptive features; The high semantic defect position information and the low semantic defect position information are fused to obtain fused defect position information; wherein the high semantic defect position information is the Euclidean distance between the defective adapted high semantic feature and the reconstructed high semantic adapted feature, and the low semantic defect position information is the Euclidean distance between the defective adapted low semantic feature and the reconstructed low semantic adapted feature; The defect discrimination network is used to take the fused defect position information as input, perform defect scoring on each pixel in the fused defect position information, and output a defect detection result.

9. An electronic device, characterized in that: include: A memory and one or more processors; the memory is coupled to the processor; wherein the memory stores computer program code, the computer program code includes computer instructions, and when the computer instructions are executed by the processor, the electronic device executes the steps of the defect detection method for industrial scenarios as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the defect detection method for industrial scenarios as described in any one of claims 1 to 7 are implemented.

Images