Method for detecting surface defects of industrial product

Abstract

The invention relates to the technical field of industrial visual inspection, and particularly provides an industrial product surface defect detection method, which comprises the following steps of: performing image-level feature extraction and defect judgment on an image of an industrial product based on a space gradient extraction module and a judgment module of a supervised learning model, and determining that the industrial product is a good product or a defective product, the space gradient extraction module is a lightweight module, and the discrimination module is provided with a residual structure enhancement layer for feature enhancement; under the condition that the industrial product is a defective product, based on a feature extraction module and a positioning module of an unsupervised learning model, pixel-level feature extraction and defect positioning are carried out on an image of the industrial product, a defect detection result is obtained, the feature extraction module is set to be a multi-branch framework with a Sobel operator, and the positioning module is set to be a multi-branch framework with a Sobel operator. And the regularization module of the positioning module is set as a bottleneck framework with weight gating. The problem that low cost, high efficiency and high precision are difficult to consider in product defect detection in the prior art is solved.

Description

Technical Field

[0001] This invention relates to the field of industrial visual inspection technology, and in particular to a method for detecting surface defects in industrial products. Background Technology

[0002] In industrial quality inspection, some production lines often have products with obvious large-area defects such as cracks and stains.

[0003] Methods for detecting defects on product surfaces include: Fully supervised learning methods such as Mask-RCNN and UNet, which require extensive and detailed pixel-level labeling, meaning the outlines of defects need to be completely drawn manually. While these methods offer high detection accuracy, the manual labeling cost is extremely high, and the re-labeling and retraining cycle is too long when new types of large-area defects appear on the production line. Unsupervised learning methods such as AutoEncoder and GANs are trained using only good product samples and detect anomalies through reconstruction errors. Although these methods do not require labeling defect samples, in actual production, due to lighting fluctuations or complex textures, they are prone to a large number of false negatives or false positives, making it difficult to meet the high-precision industrial requirements.

[0004] There is currently no effective solution to the problem that product defect detection in related technologies cannot simultaneously achieve low cost, high efficiency, and high accuracy. Summary of the Invention

[0005] The present invention provides a method for detecting surface defects in industrial products, which at least solves the problem that it is difficult to achieve low cost, high efficiency and high accuracy in product defect detection in related technologies.

[0006] This invention provides a method for detecting surface defects in industrial products, comprising: a spatial gradient extraction module based on a supervised learning model, which extracts image-level features from an image of the industrial product to obtain spatial gradient enhancement features, wherein the spatial gradient extraction module is a lightweight module; inputting the spatial gradient enhancement features into a discrimination module of the supervised learning model for defect discrimination, obtaining the image confidence level, and determining whether the industrial product is a good product or a defective product, wherein the discrimination module includes a convolutional neural network module, a residual structure enhancement layer, and a classifier arranged sequentially, and the residual structure enhancement layer is used to enhance the features output by the convolutional neural network module; in the case that the industrial product is a defective product, a feature extraction module based on an unsupervised learning model extracts pixel-level features from the image to obtain multi-channel gradient enhancement features, wherein the feature extraction module is configured with a multi-branch architecture with a Sobel operator; inputting the multi-channel gradient enhancement features into a localization module of the unsupervised learning model for defect localization, and obtaining a defect detection result, wherein the localization module includes a feature encoding module, a regularization module, and a localization unit arranged sequentially, and the regularization module is configured with a bottleneck architecture with weight gating.

[0007] Preferably, the spatial gradient extraction module includes a two-dimensional convolutional layer, a batch normalization layer, and a sigmoid linear unit arranged sequentially. Based on a supervised learning model, the spatial gradient extraction module performs image-level feature extraction on the image of the industrial product to obtain spatial gradient enhancement features, including: extracting a first gradient feature from the original features of the image based on the two-dimensional convolutional layer; performing differential processing on the first gradient feature based on the batch normalization layer to obtain a second gradient feature; and filtering and enhancing the second gradient feature based on the sigmoid linear unit to obtain the spatial gradient enhancement features.

[0008] Preferably, the spatial gradient enhancement features are input into the discrimination module of the supervised learning model for defect discrimination to obtain the image confidence level and determine whether the industrial product is a good or defective product. This includes: inputting the original features and spatial gradient enhancement features into a convolutional neural network module to obtain a first processed feature; enhancing the first processed feature based on the residual structure enhancement layer to obtain a second processed feature; inputting the second processed feature into a classifier to obtain the image confidence level; if the confidence level is greater than or equal to a preset threshold, the industrial product is a defective product; if the confidence level is less than the preset threshold, the industrial product is a good product.

[0009] Preferably, before performing image-level feature extraction on the images of industrial products using the spatial gradient extraction module based on the supervised learning model, the method further includes: performing image-level annotation processing on the sample images to obtain a training set and a test set; and training the initial model of the supervised learning model based on the training set and the test set to obtain the supervised learning model.

[0010] Preferably, when the industrial product is defective, the feature extraction module based on the unsupervised learning model performs pixel-level feature extraction on the image to obtain multi-channel gradient enhancement features, including: extracting horizontal and vertical gradient features of the image based on the Sobel operator; calculating the total gradient features based on the horizontal and vertical gradient features; and concatenating the original features, horizontal gradient features, vertical gradient features, and total gradient features of the image to obtain multi-channel gradient enhancement features.

[0011] Preferably, before performing pixel-level feature extraction on the image, the above method further includes: initializing the weights of the feature extraction module based on fixed horizontal and vertical convolutional kernels; wherein, the horizontal convolutional kernel... for: ; Vertical convolution kernel for: .

[0012] Preferably, the multi-channel gradient enhancement features are input into the localization module of the unsupervised learning model for defect localization to obtain defect detection results, including: inputting the multi-channel gradient enhancement features into the feature encoding module to obtain semantic features; inputting the semantic features into the regularization module to obtain redundancy removal features; and inputting the redundancy removal features into the localization unit to obtain defect detection results.

[0013] Preferably, the semantic features are input into the regularization module to obtain the redundancy-removing features, including: determining feature statistics and weight gating based on the semantic features; dynamically adjusting the noise intensity based on the weight gating; and determining the redundancy-removing features based on the noise intensity, semantic features, and feature statistics.

[0014] Preferably, inputting the deredundancy features into the localization unit to obtain the defect detection result includes: inputting the deredundancy features into the localization unit to generate a heatmap; performing adaptive threshold segmentation on the heatmap to obtain a binary mask; and determining the defect region based on the binary mask, wherein the defect detection result includes the defect region.

[0015] Preferably, after performing adaptive threshold segmentation on the heatmap to obtain a binarized mask, the above method further includes: generating pseudo-labels based on the binarized mask; and updating the classifier of the supervised learning model based on the pseudo-labels.

[0016] This invention provides a method for detecting surface defects in industrial products. The supervised learning model includes a lightweight feature extraction module and a residual structure enhancement layer that amplifies the output features of the convolutional neural network module. The unsupervised learning model's feature extraction module is configured with a multi-branch architecture using the Sobel operator, and its regularization module is configured with a bottleneck architecture with weight gating. This cascaded architecture, employing a supervised learning model for coarse-grained (image-level) classification and an unsupervised learning model for fine-grained (pixel-level) localization, achieves high-efficiency and high-precision defect detection with low annotation costs. This addresses the challenge of simultaneously achieving low cost, high efficiency, and high accuracy in product defect detection within related technologies. Attached Figure Description

[0017] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other embodiments based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the steps of a method for detecting surface defects in industrial products according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram illustrating the binary classification performance of the supervised learning model in an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the surface image of a good product in an embodiment of the present invention.

[0021] Figure 4 yes Figure 3 The image shown is a schematic diagram illustrating the defect location effect.

[0022] Figure 5 This is a schematic diagram of the surface image of the defective product in an embodiment of the present invention.

[0023] Figure 6 yes Figure 5 The image shown is a schematic diagram illustrating the defect location effect.

[0024] Figure 7 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0025] Embodiments of the present invention will now be described in more detail with reference to the accompanying drawings. While some embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present invention. It should be understood that the drawings and embodiments of the present invention are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0026] In industrial quality inspection, some production lines often have products with obvious large-area defects such as cracks and stains.

[0027] Methods for detecting defects on product surfaces include: Fully supervised learning methods such as Mask-RCNN and UNet, which require extensive and detailed pixel-level labeling, meaning the outlines of defects need to be completely drawn manually. While these methods offer high detection accuracy, the manual labeling cost is extremely high, and the re-labeling and retraining cycle is too long when new types of large-area defects appear on the production line. Unsupervised learning methods such as AutoEncoder and GANs are trained using only good product samples and detect anomalies through reconstruction errors. Although these methods do not require labeling defect samples, in actual production, due to lighting fluctuations or complex textures, they are prone to a large number of false negatives or false positives, making it difficult to meet the high-precision industrial requirements.

[0028] Therefore, please refer to Figure 1 As shown, the present invention provides a method for detecting surface defects in industrial products, including steps S101 to S104.

[0029] Step S101: Based on the spatial gradient extraction module of the supervised learning model, image-level feature extraction is performed on the image of the industrial product to obtain spatial gradient enhancement features. The spatial gradient extraction module is a lightweight module.

[0030] Step S102: Input the spatial gradient enhancement features into the discrimination module of the supervised learning model to perform defect discrimination, obtain the confidence level of the image, and determine whether the industrial product is a good product or a defective product. The discrimination module includes a convolutional neural network module, a residual structure enhancement layer and a classifier arranged in sequence. The residual structure enhancement layer is used to enhance the features output by the convolutional neural network module.

[0031] In step S103, when the industrial product is defective, the feature extraction module based on the unsupervised learning model performs pixel-level feature extraction on the image to obtain multi-channel gradient enhancement features. The feature extraction module is set to a multi-branch architecture with Sobel operators.

[0032] Step S104: Input the multi-channel gradient enhancement features into the localization module of the unsupervised learning model to locate defects and obtain defect detection results. The localization module includes a feature encoding module, a regularization module and a localization unit set in sequence. The regularization module is set as a bottleneck architecture with weight gating.

[0033] Specifically, the supervised learning model also includes an input layer, with a spatial gradient extraction module positioned between the input layer and the convolutional neural network module. The raw features of the industrial product image are obtained based on the input layer.

[0034] Supervised learning models require data annotation. In the embodiments of this invention, there is no need to use complex annotation methods such as bounding boxes or semantic segmentation. Instead, image-level annotation (labeling) can be performed based on the degree of severity of defects that are visible to the naked eye, which helps to reduce the cost of manual annotation.

[0035] Considering that data annotation is unnecessary when inspecting the same type of industrial product, data annotation is not a necessary step in the method provided in this embodiment. However, if the industrial product is a previously untested product type, data annotation is required to train the supervised learning model. Since only "good" needs to be labeled to correspond to good products and "bad" to correspond to defective products, even when data annotation is required, the annotation cost of the method provided in this embodiment is lower than the annotation cost required for pixel-level annotation in related methods.

[0036] The spatial gradient extraction module is a lightweight module, and the supervised learning model is a lightweight, coarse-grained, fast screening model to achieve rapid binary classification of production line images: good products and defective products. This helps to quickly screen out defective products, especially those with large-area defects. "Large-area defects" refers to the proportion of pixels in the defective area to the total number of pixels in the effective detection area of ​​the product exceeding a preset threshold, for example, exceeding 5-10%.

[0037] The residual structure enhancement layer consists of two first convolutional layers with a kernel size of 1×1, and a Gaussian error linear unit (GELU) positioned between the two first convolutional layers. By setting a residual structure enhancement layer, the supervised learning model can autonomously standardize local contrast, significantly reducing its dependence on external image preprocessing such as offline histogram equalization, and enhancing its robustness under different lighting conditions.

[0038] The supervised learning model takes the entire image to be inspected as input and outputs the confidence level of whether the corresponding industrial product is a defective product. When dealing with large areas of defects, the lightweight supervised learning model can filter out the vast majority of good products with extremely low computational consumption, quickly identifying defective products. This not only solves the problem of missed detections but also significantly reduces the amount of data required for subsequent processing.

[0039] Unsupervised learning models are based on the Transformer architecture and employ loose reconstruction loss designed for local image patches. They are reconstruction models trained using pre-selected or historically accumulated clean images.

[0040] The bottleneck architecture can also be represented as a bottleneck layer. Through a three-stage process of compressing feature dimensions, strengthening feature representation, and restoring feature dimensions, the computational complexity is significantly reduced while improving the model's ability to learn industrial defect features. In this embodiment, the bottleneck architecture is configured with weight gating, which can dynamically adjust the noise intensity based on the image contrast, thus helping to improve detection accuracy.

[0041] By incorporating the aforementioned feature extraction and regularization modules into the unsupervised learning model, the defect localization accuracy of the unsupervised learning module can be improved. When the image features of industrial products are approximately grayscale, the improvement in defect localization accuracy is particularly significant. Experimental detection images will be provided later in this embodiment for further explanation.

[0042] The method described in this embodiment is a weakly supervised or semi-supervised learning-based defect detection method. It employs a cascaded architecture of a supervised learning model for coarse-grained (image-level) classification and an unsupervised learning model for fine-grained (pixel-level) localization. This architecture achieves high-efficiency and high-precision defect detection with relatively low annotation costs. This embodiment will subsequently use the open-source model MobileNet-V3 as an example to illustrate the supervised learning model, and the open-source model DINOv3 as an example to illustrate the unsupervised learning model.

[0043] Preferably, the spatial gradient extraction module includes a two-dimensional convolutional layer, a batch normalization layer, and a sigmoid linear unit arranged sequentially.

[0044] Step S101: Based on the supervised learning model, the spatial gradient extraction module performs image-level feature extraction on the image of the industrial product to obtain spatial gradient enhancement features, including: extracting the first gradient feature from the original features of the image based on a two-dimensional convolutional layer; performing differential processing on the first gradient feature based on a batch normalization layer (BN) to obtain the second gradient feature; and filtering and enhancing the second gradient feature based on a sigmoid linear unit to obtain the spatial gradient enhancement features.

[0045] The Sigmoid Linear Unit (SiLU) is a non-linear activation function that balances gradient stability with feature selection capabilities. The "linear" in SiLU is a descriptive term emphasizing the linear structure of the input. The "non-linear" in non-linear activation functions refers to a mathematical property describing the overall mapping relationship. Therefore, the descriptions of linearity and non-linearity here are not contradictory.

[0046] The spatial gradient extraction module can specifically be That is, the kernel size of the above two-dimensional convolutional layer is 3×3.

[0047] Taking the open-source model MobileNet-V3 as an example, by configuring the spatial gradient extraction module described above, spatial gradients in contour features can be captured and enhanced preferentially before entering the convolutional layer with a stride of 2 in the default convolutional neural network module. This is crucial for identifying subtle industrial surface defects.

[0048] Preferably, step S102, which involves inputting the spatial gradient enhancement features into the discrimination module of the supervised learning model for defect discrimination to obtain the image confidence level and determine whether the industrial product is a good or defective product, includes: inputting the original features and spatial gradient enhancement features into a convolutional neural network module to obtain a first processed feature; enhancing the first processed feature based on the residual structure enhancement layer to obtain a second processed feature; inputting the second processed feature into a classifier to obtain the image confidence level; if the confidence level is greater than or equal to a preset threshold, the industrial product is a defective product; if the confidence level is less than the preset threshold, the industrial product is a good product.

[0049] Setting up a residual structure enhancement layer enables the supervised learning model to autonomously standardize local contrast, thereby enhancing the robustness of the supervised learning model under different lighting environments.

[0050] Specifically, in step S101, before the spatial gradient extraction module based on the supervised learning model performs image-level feature extraction on the image of the industrial product, the above method further includes: performing image-level annotation processing on the sample images to obtain a training set and a test set; and training the initial model of the supervised learning model based on the training set and the test set to obtain the supervised learning model.

[0051] The number of sample images (sample size) is kept to a minimum, with the training set consisting of 500 images and the test set consisting of 100 images.

[0052] Preferably, in step S103, when the industrial product is defective, the feature extraction module based on the unsupervised learning model performs pixel-level feature extraction on the image to obtain multi-channel gradient enhancement features, including: extracting horizontal and vertical gradient features of the image based on the Sobel operator; calculating the total gradient features based on the horizontal and vertical gradient features; and concatenating the original features, horizontal gradient features, vertical gradient features, and total gradient features of the image to obtain multi-channel gradient enhancement features.

[0053] The aforementioned feature extraction module is a multi-branch gradient extraction module. Taking the open-source model DINOv3 as an example, based on the ViT model (feature extractor), it uses parallel micro-branches and combines the Sobel operator to specifically extract high-frequency components. Introducing gradient anchors forces the model to focus on spatial discontinuities at low levels. The Sobel operator's inherent smoothing effect effectively filters high-frequency noise in grayscale images.

[0054] The term "micro" in "micro-branch" can refer to a convolutional layer with no more than 2 layers and a kernel size no larger than 3×3, and the total number of calculated parameters not exceeding 10% of the total number of parameters in the main branch of the feature extraction module. The main branch is used to extract the original features, while the micro-branch running parallel to the main branch is used to extract the horizontal gradient features and the vertical gradient features.

[0055] For example, an image of an industrial product is input into the feature extraction module described above, and the approximate values ​​of the partial derivatives in the horizontal and vertical directions are calculated using a 3×3 Sobel operator.

[0056] image The convolution process with the Sobel operator is as follows: ; ; In the formula, Represents the horizontal gradient response plot. Represents the vertical gradient response plot. This represents a horizontal convolution kernel used to capture edge features in the vertical direction. This represents a vertical convolution kernel used to capture edge features in the horizontal direction.

[0057] Multi-channel gradient enhancement features It can be represented as: ; ; In the formula, This indicates a channel-level concatenation operation. This represents the total gradient characteristic.

[0058] By concatenating the original features, horizontal gradient features, vertical gradient features, and total gradient features of an image, the original information can be preserved and the edge features enhanced, so that the multi-channel gradient enhancement features include the original brightness, bidirectional gradient brightness, and total gradient brightness.

[0059] Preferably, before performing pixel-level feature extraction on the image, the above method further includes: initializing the weights of the feature extraction module based on fixed horizontal and vertical convolutional kernels; wherein, the horizontal convolutional kernel... for: ; Vertical convolution kernel for: .

[0060] Since the weights are fixed, the feature extraction module does not increase the training burden.

[0061] In the actual construction and application of the model, the feature extraction module is a non-trainable layer of the unsupervised learning model, and the fixed convolutional layer uses two-dimensional convolution, with its weights determined by the above... Matrix and above The matrix is ​​initialized. The calculated gradient map is normalized to ensure that the feature distribution matches the pre-trained distribution of the open-source model DINOv3. Utilizing digital image processing priors can compensate for the problem of unsupervised learning's inability to capture weak features with small samples.

[0062] Specifically, step S104 involves inputting the multi-channel gradient enhancement features into the localization module of the unsupervised learning model to locate defects and obtain defect detection results. This includes: inputting the multi-channel gradient enhancement features into the feature encoding module to obtain semantic features; inputting the semantic features into the regularization module to obtain redundancy removal features; and inputting the redundancy removal features into the localization unit to obtain defect detection results.

[0063] Preferably, the semantic features are input into the regularization module to obtain the redundancy-removing features, including: determining feature statistics and weight gating based on the semantic features; dynamically adjusting the noise intensity based on the weight gating; and determining the redundancy-removing features based on the noise intensity, semantic features, and feature statistics.

[0064] To address the high feature redundancy in the data samples, the random deactivation regularization (Dropout) is improved by replacing it with learnable channel masking or structured sparse noise. Specifically, a weighted gating based on feature statistics is added at the bottleneck architecture to dynamically adjust the noise intensity according to the image contrast. Simple random deactivation regularization may lead to the loss of key geometric features, while dynamic noise can more accurately strike a balance between preventing identity mapping and preserving necessary reconstruction information.

[0065] Feature statistics (mean, variance, contrast, etc.) are calculated from deep semantic feature maps, and the parameters of weight gating are optimized through training based on these statistics. Both are based on semantic features.

[0066] The output (channel / region weight) of the weighted gating is a noise intensity adjustment switch: high weight (effective feature) corresponds to low noise intensity, and low weight (redundant feature) corresponds to high noise intensity.

[0067] The process involves using statistical measures to determine feature validity, weight gating to allocate and filter weights, and noise intensity to target and suppress redundant features. Ultimately, high signal-to-noise ratio (SNR) features are selected from the semantic features to eliminate redundancy. In other words, feature statistics form the basis for decision-making, weight gating acts as the adjustment mechanism, and noise intensity is the execution method, creating a closed loop of data-driven → gating adjustment → precise suppression. The resulting de-redundant features ensure both regularization effectiveness (preventing overfitting) and do not destroy the key features of surface defects in industrial products.

[0068] Preferably, inputting the deredundancy features into the localization unit to obtain the defect detection result includes: inputting the deredundancy features into the localization unit to generate a heatmap; performing adaptive threshold segmentation on the heatmap to obtain a binary mask; and determining the defect region based on the binary mask, wherein the defect detection result includes the defect region.

[0069] The unsupervised learning model calculates the anomaly map between the current image and the reconstructed image, and generates a pixel-level defect mask (i.e., the aforementioned binarized mask) through threshold segmentation, thereby accurately marking the specific location and shape of the defect.

[0070] Preferably, after performing adaptive threshold segmentation on the heatmap to obtain a binarized mask, the above method further includes: generating pseudo-labels based on the binarized mask; and updating the classifier of the supervised learning model based on the pseudo-labels.

[0071] Updating the classifier of a supervised learning model using pseudo-masks can help improve detection accuracy.

[0072] In summary, the beneficial effects of the technical solution provided in this embodiment include, but are not limited to: image-level classification and labeling of good / bad only a small number of samples, eliminating the need for time-consuming and laborious pixel-level annotation, saving more than 90% of manpower costs, i.e., significantly reducing annotation costs; the use of a lightweight supervised learning network specifically trained for large-area defects compensates for the shortcomings of pure unsupervised algorithms that are prone to missed detection when features are not obvious, ensuring the accuracy of targeted screening, i.e., improving the detection rate and reducing missed detections; the lightweight network, as a pre-filter, has extremely low computational cost and can quickly process massive amounts of data on the production line, performing complex unsupervised localization calculations only on suspected defective products that have passed the screening, greatly improving the overall operating efficiency (FPS) of the system, i.e., balancing speed and accuracy; even without pixel-level training data, the visualization of defect areas is still achieved through the unsupervised module, facilitating subsequent analysis of defect causes by process personnel, i.e., possessing pixel-level localization capabilities.

[0073] For the method provided in this embodiment, this embodiment also provides corresponding experimental descriptions, as follows.

[0074] Five hundred silicon wafer images were collected, including 260 images with large-area cracks. These 260 images with large-area cracks were labeled "NG-coarse-grained," while the remaining 240 images were labeled "NG-fine-grained" (no labeling required). The supervised learning model provided in this embodiment was trained for binary classification using these 500 silicon wafer images. Additionally, the unsupervised learning model provided in this embodiment was trained using 300 flawless sample images to establish a good-quality feature library.

[0075] The binary classification performance of the above supervised learning model is as follows: Figure 2 As shown, Figure 2 The two-dimensional performance curve plotted in the figure is the Receiver Operating Characteristic Curve (ROC), with the horizontal axis representing the false positive rate (FPR) and the vertical axis representing the true positive rate (TPR). AUC is an abbreviation for Area Under the ROC Curve, and a value of 0.991 represents the percentage of the area enclosed by the ROC curve and the coordinate axes, which can also be expressed as AUROC = 99.1%.

[0076] In the field of industrial defect detection, an AUC of 95% or higher is considered excellent. An AUC of 99.1% indicates an extremely high true positive rate and an extremely low false positive rate. An extremely high true positive rate means that the supervised learning model can effectively identify genuine defective products with a very low false negative rate, preventing defective products from entering the market. An extremely low false positive rate means that the supervised learning model rarely misclassifies good products as defective products, resulting in a very low false positive rate and reducing the cost of incorrectly rejecting good products.

[0077] In other words, AUC=0.991 means that no matter which confidence threshold is chosen, the above supervised learning model can maintain a balance between high recognition rate and low false positive rate, adapting to the priority requirements of industrial products on different production lines for missed detection and false positive detection.

[0078] Based on the above unsupervised learning model obtained through training, Figure 3 The defect location was determined by analyzing the surface image of the compliant (good) product shown. Please refer to the resulting image. Figure 4 As shown.

[0079] Based on the above unsupervised learning model obtained through training, Figure 5 The defective product (substandard product) surface image is used to locate the defect. Please refer to the resulting image for the correct illustration. Figure 6 As shown, the cracked area is highlighted in red.

[0080] Combination Figure 4 and Figure 6 As can be seen, the defect localization effect diagram of the good product does not have obvious high-heat areas except for the inherent accompanying area at the edge, while the defect localization effect diagram of the defective product clearly outlines the specific direction and coverage area of ​​the crack, indicating that the unsupervised learning model provided in this embodiment has a good defect localization function.

[0081] In summary, the method provided by the embodiments of the present invention combines a lightweight supervised classification network (for image-level screening) with unsupervised reconstruction, which can achieve pixel-level defect detection using a small number of image-level labels (data annotations), resulting in low detection cost and high detection efficiency. It can also use a lightweight network to prioritize screening out large-area obvious defects, solving the problem of poor robustness of unsupervised algorithms under different amplitude texture variations, and achieving high detection accuracy.

[0082] The present invention also provides a non-transitory machine-readable medium storing a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform a method according to an embodiment of the present invention.

[0083] The present invention also provides a computer program product, including a computer program, wherein the computer program, when executed by a computer's processor, is used to cause the computer to perform the method of the embodiments of the present invention.

[0084] This invention also provides an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor. The memory stores a computer program executable by the at least one processor, which, when executed by the at least one processor, causes the electronic device to perform the method of this invention.

[0085] refer to Figure 7 This is a structural block diagram of an electronic device, either a server or a client, according to an embodiment of the present invention. It is an example of a hardware device that can be applied to various aspects of the present invention. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0086] like Figure 7 As shown, the electronic device includes a computing unit 701, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 702 or a computer program loaded into a random access memory (RAM) 703 from a storage unit 708. The RAM 703 may also store various programs and data required for the operation of the electronic device. The computing unit 701, the ROM 702, and the RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0087] Multiple components in the electronic device are connected to I / O interface 705, including: input unit 706, output unit 707, storage unit 708, and communication unit 709. Input unit 706 can be any type of device capable of inputting information into the electronic device. Input unit 706 can receive input digital or character information and generate key signal inputs related to user settings and / or function control of the electronic device. Output unit 707 can be any type of device capable of presenting information and may include, but is not limited to, a display, speaker, video / audio output terminal, vibrator, and / or printer. Storage unit 708 may include, but is not limited to, disks and optical discs. Communication unit 709 allows the electronic device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks, and may include, but is not limited to, modems, network cards, infrared communication devices, and / or wireless communication transceivers, such as Bluetooth devices, WiFi devices, WiMax devices, cellular communication devices, and / or the like.

[0088] The computing unit 701 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, CPUs, graphics processing units (GPUs), various special-purpose artificial intelligence (AI) computing units, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present invention can be implemented as computer programs tangibly contained in a machine-readable medium, such as storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on an electronic device via ROM 702 and / or communication unit 709. In some embodiments, the computing unit 701 can be configured to perform the methods described above by any other suitable means (e.g., by means of firmware).

[0089] Computer programs for implementing the methods of embodiments of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0090] In the context of embodiments of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, or infrared systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0091] It should be noted that the term "comprising" and its variations used in the embodiments of this invention are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "a plurality" mentioned in the embodiments of this invention are illustrative and not restrictive, and those skilled in the art should understand that unless explicitly indicated otherwise in the context, they should be understood as "one or more". The descriptions of terms such as "first", "second", etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of indicated technical features.

[0092] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this invention are all information and data authorized by the user or fully authorized by all parties.

[0093] The steps described in the method embodiments provided by the present invention can be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of protection of the present invention is not limited in this respect.

[0094] The term "embodiment" in this specification refers to a specific feature, structure, or characteristic described in connection with an embodiment that may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily imply the same embodiment, nor does it imply independence or alternativeity from other embodiments. The various embodiments in this specification are described in a related manner, with reference to each other for similar or identical parts. In particular, for apparatus, device, and system embodiments, since they are substantially similar to method embodiments, the description is relatively simple, and relevant details are referred to in the description of the method embodiments.

[0095] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of protection. It should be noted that those skilled in the art can make various modifications and improvements without departing from the inventive concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A method for detecting surface defects in industrial products, characterized in that, include: The spatial gradient extraction module based on the supervised learning model performs image-level feature extraction on images of industrial products to obtain spatial gradient enhancement features. The spatial gradient extraction module is a lightweight module. The spatial gradient enhancement features are input into the discrimination module of the supervised learning model for defect discrimination, and the confidence score of the image is obtained to determine whether the industrial product is a good product or a defective product. The discrimination module includes a convolutional neural network module, a residual structure enhancement layer and a classifier arranged in sequence. The residual structure enhancement layer is used to enhance the features output by the convolutional neural network module. In the case that the industrial product is defective, the feature extraction module based on the unsupervised learning model performs pixel-level feature extraction on the image to obtain multi-channel gradient enhancement features. The feature extraction module is set to a multi-branch architecture with Sobel operators. The multi-channel gradient enhancement features are input into the localization module of the unsupervised learning model for defect localization to obtain defect detection results. The localization module includes a feature encoding module, a regularization module and a localization unit arranged in sequence. The regularization module is set as a bottleneck architecture with weight gating.

2. The method according to claim 1, characterized in that, The spatial gradient extraction module includes a two-dimensional convolutional layer, a batch normalization layer, and a sigmoid linear unit arranged sequentially. The spatial gradient extraction module based on the supervised learning model performs image-level feature extraction on images of industrial products to obtain spatial gradient enhancement features, including: The first gradient feature is extracted from the original features of the image based on the two-dimensional convolutional layer; The first gradient feature is differentiated based on the batch normalization layer to obtain the second gradient feature; The second gradient feature is filtered and enhanced based on the S-shaped linear unit to obtain the spatial gradient enhancement feature.

3. The method according to claim 2, characterized in that, The spatial gradient enhancement features are input into the discrimination module of the supervised learning model for defect discrimination, and the confidence score of the image is obtained to determine whether the industrial product is a good or defective product, including: The original features and the spatial gradient enhancement features are input into the convolutional neural network module to obtain the first processed features; Based on the residual structure enhancement layer, the first processed feature is enhanced to obtain the second processed feature; The second processed feature is input into the classifier to obtain the confidence score of the image. If the confidence score is greater than or equal to a preset threshold, the industrial product is a defective product; if the confidence score is less than the preset threshold, the industrial product is a good product.

4. The method according to claim 1, characterized in that, Before performing image-level feature extraction on images of industrial products, the method, based on a supervised learning model-based spatial gradient extraction module, further includes: Image-level annotation is performed on the sample images to obtain the training set and the test set; The initial model of the supervised learning model is trained based on the training set and the test set to obtain the supervised learning model.

5. The method according to claim 1, characterized in that, In the case where the industrial product is defective, the feature extraction module based on the unsupervised learning model performs pixel-level feature extraction on the image to obtain multi-channel gradient enhancement features, including: Based on the Sobel operator, extract the horizontal and vertical gradient features of the image; Calculate the total gradient feature based on the horizontal gradient feature and the vertical gradient feature; The original features, horizontal gradient features, vertical gradient features, and total gradient features of the image are concatenated to obtain the multi-channel gradient enhancement features.

6. The method according to claim 5, characterized in that, Before performing pixel-level feature extraction on the image, the method further includes the following: (Based on an unsupervised learning model-based feature extraction module) The weights of the feature extraction module are initialized based on fixed horizontal and vertical convolution kernels; Among them, the horizontal convolution kernel for: ； Vertical convolution kernel for: 。 7. The method according to claim 1, characterized in that, The multi-channel gradient enhancement features are input into the localization module of the unsupervised learning model for defect localization, resulting in defect detection results, including: The multi-channel gradient enhancement features are input into the feature encoding module to obtain semantic features; The semantic features are input into the regularization module to obtain the deredundancy features; The deredundancy features are input into the localization unit to obtain the defect detection result.

8. The method according to claim 7, characterized in that, The semantic features are input into the regularization module to obtain deredundancy features, including: Based on the semantic features, feature statistics and weight gating are determined; The noise intensity is dynamically adjusted based on the aforementioned weighted gating. The redundancy removal features are determined based on the noise intensity, the semantic features, and the feature statistics.

9. The method according to claim 7, characterized in that, The deredundancy features are input into the localization unit to obtain the defect detection result, including: The deredundancy features are input into the positioning unit to generate a heat map; The heatmap is subjected to adaptive threshold segmentation to obtain a binarized mask; The defect region is determined based on the binarized mask, wherein the defect detection result includes the defect region.

10. The method according to claim 9, characterized in that, After performing adaptive threshold segmentation on the heatmap to obtain a binarized mask, the method further includes: Pseudo-labels are generated based on the binarized mask; The classifier of the supervised learning model is updated based on the pseudo-labels.

Images