Defect detection methods, media, and equipment based on feature reconstruction

Abstract

This application provides a defect detection method, medium, and device based on feature reconstruction, belonging to the field of image processing technology. The method includes: acquiring a test image; performing multi-scale downsampling processing on the test image to form multi-scale images containing the test image; independently reconstructing features for each scale image, obtaining a corresponding reconstructed image for each scale image; calculating the error between each scale image and the corresponding reconstructed image to obtain a corresponding error image; identifying the defect type and probability distribution of the test image; determining the optimized weights of the error images at each scale based on the defect type and probability distribution; re-weighting and fusing the multi-scale error images according to the optimized weights to obtain an optimized fusion difference map; and performing defect identification based on the optimized fusion difference map. This application can improve the accuracy of defect detection.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a defect detection method, medium, and device based on feature reconstruction. Background Technology

[0002] In the development of intelligent manufacturing and industrial automation, surface defect detection is a core component of the quality control system, directly determining product quality and production efficiency. As industrial products become increasingly sophisticated and complex, higher demands are placed on the accuracy, speed, and generalization capabilities of defect detection. Traditional detection methods are no longer adequate for the needs of modern production lines.

[0003] In recent years, the development of deep learning technology has provided new solutions for industrial defect detection. Unsupervised defect detection methods, which only require normal samples for model training and do not require scarce defect-labeled samples, have become a research focus for adapting to actual industrial production scenarios. Among them, the unsupervised detection approach based on feature reconstruction has been widely used, such as existing technologies CN121329966A and CN114332008A. The core principle is to learn the features and image patterns of normal samples through the model, generate significant errors in the reconstruction process of abnormal regions, and then realize defect identification and localization based on the analysis of reconstruction errors. Related technologies have also been explored and applied to some extent in industrial inspection scenarios.

[0004] However, as industrial product structures become increasingly complex and the textures of defect features become more intricate and diverse, the accuracy of current feature-reconstruction-based defect detection methods in identifying a wide variety of defect features still needs improvement. Summary of the Invention

[0005] The purpose of this application is to provide a defect detection method, medium, and device based on feature reconstruction to solve at least one of the above-mentioned technical problems.

[0006] In a first aspect, this application provides a defect detection method based on feature reconstruction, the method comprising: Acquire the image to be tested; The image to be tested is downsampled at multiple scales to form an image at multiple scales containing the image to be tested; Each scale image is reconstructed independently, resulting in a corresponding reconstructed image for each scale image. Calculate the error between the image at each scale and the corresponding reconstructed image to obtain the corresponding error image; Identify the defect type and probability distribution of the image under test; The optimized weights for the error images at each scale are determined based on the defect type and probability distribution. The error images at multiple scales are re-weighted and fused according to the optimized weights to obtain an optimized fusion difference map, and defect identification is performed based on the optimized fusion difference map.

[0007] Optionally, identifying the defect type and probability distribution of the image to be tested includes: Error images at multiple scales are weighted and fused according to preset initial weights to obtain an initial fusion difference map; Identify the defect types and probability distributions of the fused difference map.

[0008] Optionally, the defect identification based on the optimized fusion difference map includes: Based on the defect type and probability distribution, a defect threshold is determined. The optimized fusion difference map is then segmented according to the determined defect threshold to generate a binary defect mask. Defect identification is then performed based on the binary defect mask.

[0009] Optionally, the step of independently reconstructing features for each scale image, resulting in a corresponding reconstructed image for each scale image, includes: Each scale image is input into a pre-defined multi-layer reconstruction network model, with each scale image configured with an independent encoder-decoder sub-network; The corresponding encoder-decoder subnetwork performs feature extraction and image reconstruction on the corresponding scale image to obtain the reconstructed image of the corresponding scale.

[0010] Optionally, the encoder in the encoder-decoder sub-network is built based on the ResNet network. The construction process of the encoder includes: removing the global average pooling layer and fully connected layer of the ResNet network, and retaining the convolutional layer part; replacing the last 3×3 standard convolution in each residual block group of the ResNet network with a dilated convolution to obtain an encoder adapted to scale image feature extraction.

[0011] Optionally, the decoder adopts a U-Net-style skip connection structure, and the process of reconstructing the encoder output features by the decoder includes: The resolution of the feature map corresponding to the scale image is gradually restored through upsampling operations, and in each upsampling stage, the decoder features are spliced ​​and fused with the features of the corresponding layer of the encoder. The spliced ​​and fused features are subjected to convolutional dimensionality reduction and feature extraction to finally generate a reconstructed image of the same size as the input image.

[0012] Optionally, the multi-layer reconstruction network model is trained using a sample image set to obtain a trained multi-layer reconstruction network model. The joint loss function used during training is a weighted sum of the pixel reconstruction loss function, the feature consistency loss function, and the gradient preservation loss function.

[0013] Optionally, the defect types include texture defects, structural fracture defects, local peeling, or scratch defects; determining the optimization weights of the error image at each scale based on the defect type and probability distribution includes: The optimized weight values ​​for error images at high-resolution scales are positively correlated with the probability of texture defects; and / or The optimized weight values ​​for error images at medium resolution scales are positively correlated with the probability of structural fracture defects; and / or The optimized weight values ​​for error images at low resolution scales are positively correlated with the probability of local peeling or scratch defects.

[0014] In a second aspect, this application provides a computer-readable storage medium storing executable instructions that, when executed by a processor, cause the processor to perform the method described in any embodiment of this application.

[0015] In a third aspect, this application provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, which, when executed by the one or more processors, cause the one or more processors to perform the method as described in any embodiment of this application.

[0016] The feature reconstruction-based defect detection method, medium, and device in this application construct multiple reconstructed images with multi-scale feature reconstruction as the core. After initially identifying the defect type and probability distribution of the image to be tested, the fusion weights of the multiple reconstructed images are optimized. This allows the fusion weights to be adaptively adjusted according to the initially identified defect situation, resulting in optimized weights. This avoids the technical problems of fixed weights and poor adaptability to multiple defect types in traditional feature reconstruction defect detection methods. It can achieve high-precision, high-robustness, and high-adaptability defect detection in industrial vision inspection scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0018] Figure 1 This is a flowchart illustrating a feature-reconstruction-based defect detection method in one embodiment; Figure 2 This is a schematic diagram of the structure of an electronic device in one embodiment. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] All terms used in this application (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0021] For example, the terms "first," "second," etc., used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from another element.

[0022] For example, the terms "comprising" or "including" used in this application indicate the presence of features, steps, operations and / or components, but do not exclude the presence or addition of one or more other features, steps, operations or components.

[0023] This application provides a defect detection method based on feature reconstruction, such as... Figure 1 As shown, the method includes: Step 110: Obtain the image to be tested.

[0024] In this embodiment, the image to be tested refers to a visual image containing the industrial product to be inspected, which is acquired by an industrial vision acquisition device under a standardized shooting environment. The acquisition device can be an industrial area scan camera, a line scan camera, etc., and the shooting environment must match the lighting, shooting distance, and angle standards of industrial inspection to avoid image noise introduced by environmental interference.

[0025] Taking the detection of surface defects on mobile phone glass covers as an example, a 2-megapixel industrial area array camera can be used to vertically photograph the mobile phone glass cover under uniform diffused light, and a 512×512 pixel RGB color image to be tested can be acquired, with 3 image channels.

[0026] Step 120: Perform multi-scale downsampling processing on the image to be tested to form an image at multiple scales containing the image to be tested.

[0027] In this embodiment, downsampling refers to reducing the pixel size of an image by a certain factor while maintaining the overall characteristics of the image, thereby reducing the amount of image data and achieving feature representation at different scales. The multi-scale images include the image to be tested, as well as a set of supplementary images at corresponding scales obtained by downsampling the image to be tested at different factors. That is, they include the original scale image and images at each downsampled scale. Different scale images are used to capture different types of defect features; for example, high-resolution scales preserve texture details, while low-resolution scales enhance global structural features.

[0028] Optionally, the downsampling factor can be set to three factors: 2x, 4x, and 8x. Combined with the original-scale image to be tested, the resulting multi-scale images include a first-scale image of the original image to be tested, a second-scale image downsampled by 2x, a third-scale image downsampled by 4x, and a fourth-scale image downsampled by 8x. Understandably, the downsampling factor can also be set to other suitable factors, and the number of multi-scale images can also be a correspondingly appropriate number of images.

[0029] Optionally, this application may employ bilinear interpolation downsampling. This method calculates and fills new pixel values ​​by linearly weighting adjacent pixels, ensuring that the downsampled image is smooth and free of checkerboard effects, and avoiding artifacts introduced by downsampling. Specifically, the calculation logic for bilinear interpolation 2x downsampling is as follows: a 2×2 pixel region of the original image is mapped to one pixel of the downsampled image. This pixel value is the weighted average of the original 2×2 pixels, with the weights determined by the distance between pixels. The same logic is executed sequentially based on the previous downsampling results for 4x and 8x downsampling.

[0030] Using the original image to be tested as the first-scale image (size 512×512), bilinear interpolation is performed sequentially to downsample it by 2x, 4x, and 8x, resulting in the second-scale image (256×256), the third-scale image (128×128), and the fourth-scale image (64×64). The final multi-scale image set is {512×512, 256×256, 128×128, 64×64}. The pixel positions of each scale image correspond one-to-one with the original image to be tested, ensuring the scale consistency of subsequent feature reconstruction.

[0031] Step 130: Perform feature reconstruction independently on each scale image, and obtain a corresponding reconstructed image for each scale image.

[0032] In this embodiment, feature reconstruction refers to the process of extracting features from a scale image and then restoring the extracted features to an image of the same size as the input image. Its core is the accurate reconstruction of normal regions of the input image, while introducing reconstruction deviations for abnormal regions. The reconstructed image is the output image after feature extraction and reconstruction of the input scale image. The reconstructed image has the exact same size and number of channels as the input image at the corresponding scale, forming the basis for subsequent calculations of reconstruction errors. In this embodiment, feature extraction and image reconstruction are performed independently for each scale image, without interference, ensuring that the reconstruction accuracy of images at different scales is adapted to their scale features.

[0033] Step 140: Calculate the error between the image at each scale and the corresponding reconstructed image to obtain the corresponding error image.

[0034] In this embodiment, the error image is used to reflect the degree of error between the reconstructed image and the image at the corresponding scale. The degree of error is reflected by the size of the pixel value in the error image. The larger the pixel value, the higher the probability that the location is a defect.

[0035] The error is calculated using a pixel-by-pixel absolute error method, performing error calculations between the input image and the corresponding reconstructed image at each scale. The calculation formula is as follows: Where k represents the corresponding scale image and reconstructed image number, for example, it can be 1 to 4, and (x, y) are the pixel coordinates. Let be the pixel value at (x, y) of the image at scale k. Let be the pixel value at (x,y) of the reconstructed image at scale k. This represents the pixel-by-pixel absolute error at that location.

[0036] For an RGB color image, the absolute error per pixel is calculated for the R, G, and B channels respectively, and the average error of the three channels is taken as the final error value of the pixel. After calculating the error values ​​of all pixels in the scaled image, the error values ​​are mapped to gray values ​​of 0-255 to generate an error image of the same size as the scaled image.

[0037] In this embodiment, taking the above four scale images as examples, after error calculation, four error images of 512×512, 256×256, 128×128, and 64×64 are obtained respectively. The first scale (512×512) error image clearly retains the error features of minor defects such as microcracks and fine scratches, while the fourth scale (64×64) error image highlights the error features of structural defects such as edge breakage and large-scale peeling.

[0038] To ensure scale consistency in subsequent error image fusion, the error images at the second, third, and fourth scales were all upsampled to the 512×512 size of the first scale using bilinear interpolation, resulting in four standardized error images of 512×512.

[0039] Step 150: Identify the defect type and probability distribution of the image to be tested.

[0040] In this embodiment, defect types can be categorized into several types, including texture defect types (such as microcracks and fine scratches, which rely on high-resolution scale capture), structural fracture defect types (such as edge fractures and structural damage, which rely on medium-resolution scale capture), and local peeling or scratch defect types (such as glass layer peeling and long scratches, which rely on low-resolution scale capture). The probability distribution refers to the probability value of each type of defect existing in the image under test, ranging from 0 to 1. The sum of the probability values ​​of all defect types is 1. The higher the probability value, the greater the likelihood that the image under test contains that type of defect.

[0041] In one embodiment, the electronic device has a preset defect recognition model, which can directly use the image to be tested as the input of the model to directly identify the corresponding defect type and probability distribution based on the image to be tested; in addition, it can also use multiple error images as the input of the model to identify the corresponding defect type and probability distribution through the multiple error images.

[0042] This defect recognition model can be a lightweight convolutional neural network model. For example, using various defect samples from mobile phone glass covers as the training set, the model learns the feature patterns of different defect types. The output layer uses the Softmax activation function to ensure that the sum of the output probabilities is 1. This lightweight convolutional neural network model can quickly and initially obtain the probability distribution of various defect types in the image under test. For instance, the defect recognition model performs feature extraction and defect type determination on the fused input image and / or the image under test, outputting a probability of 0.6 for texture defects, 0.1 for structural fracture defects, and 0.3 for local scratch defects in the image under test.

[0043] By performing preliminary identification of defect type and defect probability based on the image under test and / or multiple error images, the defect situation of the image under test can be roughly determined, thus providing a reference for subsequent defect confirmation.

[0044] Step 160: Determine the optimized weights of the error images at each scale based on the defect type and probability distribution.

[0045] In this embodiment, the initially identified defect types and probability distributions are used to determine the optimized weights for error images at each scale, so that each error image can be fused in a targeted manner based on the optimized weights, thereby improving the accuracy of subsequent defect type identification.

[0046] Understandably, for the multiple error images obtained, the error images at different scales have different scale adaptability to represent different types of defects. Based on the calculated defect probability, the proportion and dominance of various defects in the image under test can be quantified, so that the calculated optimized weights can make the matching between defect features and scale representation capabilities more accurate. By adaptively adjusting the contribution of each scale error image in the fusion (optimized weights), the error features of dominant defects can be strengthened and the interference of secondary features or background noise can be suppressed.

[0047] The error images obtained after feature reconstruction from high, medium, and low resolution images generated by multi-scale downsampling exhibit scale-specific capabilities in capturing defects. For example, considering four scales of error images—original scale, 2x downsampled, 4x downsampled, and 8x downsampled—the high-resolution error image can be either the original scale or a 2x downsampled error image; the medium-resolution error image is a 4x downsampled error image; and the low-resolution error image is an 8x downsampled error image.

[0048] High-resolution scale error images preserve pixel-level texture details and have the strongest representation ability for texture defects (such as microcracks, fine scratches, and pinholes, characterized by small size, significant texture features, and obvious pixel-level differences). They can clearly reflect the key features of small defects such as edges and contours. Medium-resolution scale error images balance detail preservation and global structure. They have the best representation ability for structural fracture defects (such as local edge fractures and product structural damage, characterized by abrupt changes in local morphology and medium-scale structural anomalies). They can accurately capture the structural change patterns of defects. Low-resolution scale error images lose local details but enhance the global morphology and overall structural features of the image. They have the strongest representation ability for local peeling / long-scratch defects (such as large-scale peeling of glass layers and long-distance scratches, characterized by large scale and significant global morphological features). They can effectively distinguish the global structural differences between these defects and normal areas.

[0049] In this embodiment, a pre-established correspondence between defect types, their corresponding probabilities, and dominant scales is established. For example, the dominant scales for texture defects are the first and second scales (high / medium resolution, capturing details); the dominant scales for structural fracture defects are the second and third scales (medium resolution, capturing local structures); and the dominant scales for local peeling / scratching defects are the third and fourth scales (low resolution, capturing global morphology). Based on the defect probability distribution output in step 150, the initial weights of the corresponding dominant scales are amplified proportionally, while the weights of the non-dominant scales are reduced proportionally. Finally, normalization is performed to obtain the optimized weights for the error images at each scale.

[0050] Optionally, for optimizing weights, the optimized weight value of the error image at a high resolution scale is positively correlated with the probability of texture defects; the optimized weight value of the error image at a medium resolution scale is positively correlated with the probability of structural fracture defects; and the optimized weight value of the error image at a low resolution scale is positively correlated with the probability of local peeling or scratch defects.

[0051] If the image under test is mainly characterized by texture defects, the optimization direction is to increase the weight of the high-resolution scale and appropriately reduce the weight of the medium / low-resolution scale, so that the small defect features of the high-resolution error image dominate the fusion process. If the image is mainly characterized by structural fracture defects, the optimization direction is to increase the weight of the medium-resolution scale to enhance the error features of structural anomalies. If the image is mainly characterized by local peeling / long scratch defects, the optimization direction is to increase the weight of the low-resolution scale to highlight the differences in defects in the global structure. If the image is characterized by mixed defects, the optimization direction is to adjust the weights of multiple dominant scales simultaneously according to the proportion of each type of defect, so as to enhance the features of multiple defects at the same time.

[0052] Taking the calculated texture defect of 0.6, structural fracture defect of 0.1, and local peeling / scratching defect of 0.3 as examples, initial weights for error images of different scales are preset, such as the initial weights W=[0.4,0.3,0.2,0.1] for the four scales of error images, corresponding to scales 1-4 respectively. The weights of the dominant scales are amplified according to probability and normalized to obtain the optimized weights w': the proportion of texture defects is 0.6, so the initial weights of scales 1 and 2 are multiplied by 0.6 / 0.3 (probability proportion / average probability) to get the weights of scale 1 as 0.8 and scale 2 as 0.6; the proportion of structural fracture defects is 0.1, so the weight of scale 3 remains at 0.2; the proportion of local peeling / scratching defects is 0.3, so the weight of scale 4 is multiplied by 0.3 / 0.3 to get 0.1; the adjusted temporary weights are W=[0.8,0.6,0.2,0.1]. After normalizing the temporary weights, we obtain the optimized weights W'=[0.47,0.35,0.12,0.06]. Compared with the basic weights, the optimized weights significantly improve the weights of the advantageous scales 1 and 2 of the texture defects, making them more suitable for the defect features of the image under test.

[0053] Step 170: Re-weight and fuse the error images at multiple scales according to the optimized weights to obtain an optimized fusion difference map, and perform defect identification based on the optimized fusion difference map.

[0054] In this embodiment, the image obtained by weighting and fusing the error images at each scale pixel by pixel according to optimized weights is the fusion difference map, which can be a grayscale image. The fusion difference map enhances the error features of the main defect types in the current image under test and suppresses the interference of secondary features, resulting in higher contrast between the defect area and the normal area.

[0055] You can follow the formula To achieve integration, among which, Here are the pixel coordinates of the image, and N represents the number of scale images / error images. For example, if N is 4, it represents the original scale error image (error image of the first scale), the 2x downsampled error image (error image of the second scale), the 4x downsampled error image (error image of the third scale), and the 8x downsampled error image (error image of the fourth scale), respectively. This represents the optimized weights for the error image corresponding to the k-th scale. This represents the error image corresponding to the k-th scale. This indicates an optimized fusion difference diagram. To optimize the pixel values ​​at (x,y) in the fusion difference map.

[0056] In this embodiment, threshold segmentation can be used for defect identification. Specifically, the optimized fusion difference map is subjected to threshold segmentation processing, dividing the pixel values ​​in the image into defect regions and normal regions, generating a binary defect mask (an image containing only 0 and 1 pixel values). This threshold can be adaptively set based on the detection results.

[0057] Taking the detection of mobile phone glass covers as an example, adaptive threshold segmentation (based on local neighborhood Gaussian weighting) is selected. This method can adapt to subtle differences in local illumination of the image and automatically segment each pixel. Calculate the specific segmentation threshold t(x,y), and the segmentation rule is: if ( If a pixel has a value of 1, it is marked as 1 (defect area); otherwise, it is marked as 0 (normal area). The final generated binary defect mask is the same size as the original image to be tested. The white area with a pixel value of 1 accurately corresponds to the location, shape, and size of the defect in the mobile phone glass cover. The defect area can be directly outlined in the original image to be tested using this binary defect mask to complete defect identification. If it is necessary to quantify the detection results, parameters such as the area, perimeter, and longest diameter of the defect area can also be calculated to match industrial quality judgment standards.

[0058] The feature reconstruction-based defect detection method in this application constructs multiple reconstructed images based on multi-scale feature reconstruction. After initially identifying the defect type and probability distribution of the image under test, the fusion weights of the multiple reconstructed images are optimized. This allows the fusion weights to be adaptively adjusted according to the initially identified defects, resulting in optimized weights. This avoids the technical problems of fixed weights and poor adaptability to multiple defect types in traditional feature reconstruction defect detection methods. It can achieve high-precision, high-robustness, and high-adaptability defect detection in industrial vision inspection scenarios. Specifically, it has the following technical effects: 1. Multi-scale independent reconstruction to achieve full-type defect coverage: By downsampling the test image at multiple scales and configuring an independent feature reconstruction network for each scale, the high-resolution scale captures the detailed features of texture defects such as microcracks and fine scratches, while the low-resolution scale captures the global features of structural defects such as edge breakage and local peeling. This solves the problem that traditional single-scale reconstruction cannot take into account both micro and macro defects, and achieves full coverage of multi-type and multi-scale defects.

[0059] 2. Defect type adaptive weight optimization to improve detection accuracy: Based on the probability distribution of defect types in the image under test, the weights of error images at each scale are adaptively adjusted, so that the dominant scale error features of the main defect types dominate in the fusion, significantly enhancing the contrast between the defect area and the normal area, and effectively reducing the false negative rate of small defects and the localization error of complex defects.

[0060] 3. Step-by-step fusion and identification to improve the robustness of the method: First, preliminary identification of defect types is carried out, and then the weights are optimized and re-fused based on the identification results to form a closed loop of "preliminary identification-weight optimization-precise detection". This allows the method to adaptively adjust according to the defect features of different test images, effectively adapting to industrial real-world scenarios with uneven lighting and complex defect morphologies, and reducing the interference of background texture and image noise on the detection results.

[0061] In one embodiment, identifying the defect type and probability distribution of the image under test includes: weighting and fusing error images of multiple scales according to preset initial weights to obtain an initial fusion difference map; and identifying the defect type and probability distribution of the fusion difference map.

[0062] In this embodiment, the electronic device pre-sets an initial weight for the error image at each scale, and the sum of the initial weights for all scales is 1. This can be flexibly set according to the general scenario of industrial inspection. For example, the preset initial weights W for the error images at the above four scales are [0.4, 0.3, 0.2, 0.1].

[0063] Similarly, a pixel-by-pixel weighted summation method can be used to calculate the error images at all scales. (such as the above) The initial fusion difference map E is calculated pixel by pixel. p The pixel value is calculated using the following formula: . Represents the pixel coordinates of the image. This represents the initial weights of the error image corresponding to the k-th scale. This represents the error image corresponding to the k-th scale. This represents the initial fusion difference map. The pixel value at (x,y) is the initial fused difference map.

[0064] The values ​​of all pixels in the initial fusion difference map are calculated sequentially, and the pixel values ​​are mapped to a grayscale range of 0-255 to finally generate an initial fusion difference map. In this image, the pixel values ​​of the microcrack (texture defect) region are significantly higher than those of the normal region due to the high-resolution scale weight. The local peeling (low-resolution advantage defect) region also retains the basic error features, providing a complete error feature basis for subsequent defect type identification.

[0065] This embodiment uses a lightweight convolutional integral class network to identify the defect type and probability distribution of the initial fusion difference map. This network is a pre-trained dedicated defect identification model. Taking an initial fusion difference map of 512×512 as an example, the specific implementation process of the model is as follows: 1. Recognition Model Preprocessing: The initial fusion difference image (single-channel grayscale image) of 512×512 is standardized, and the pixel values ​​are normalized from 0-255 to 0-1 to eliminate the interference of pixel value dimensions on model recognition. At the same time, the image size is adjusted to the standard input size of 256×256 to ensure that it is consistent with the input dimension during training.

[0066] 2. Model Input and Feature Extraction: The preprocessed initial fusion difference map is input into a lightweight convolutional classification network. This network can be built on the MobileNetV2 architecture and extracts multi-level defect features of the image through depthwise separable convolution, including pixel-level detail features (adapted to texture defect recognition), local structural features (adapted to structural fracture defect recognition), and global morphological features (adapted to local peeling defect recognition). This avoids the problem of excessive computation in traditional convolutional networks and meets the needs of real-time industrial inspection.

[0067] 3. Defect Type and Probability Distribution Output: The network's output layer uses the Softmax activation function to map the extracted features to probability values ​​for each type of defect, ensuring that the sum of all probability values ​​is 1. The output is a one-dimensional vector containing the probabilities of the three defect types. ],in Let be the defect probability of the i-th defect type. Taking m=3 as an example, then... This represents the probability of texture defects. This represents the probability of structural fracture defects. This represents the probability of localized peeling / scratching defects.

[0068] For example, after model recognition, the output probability distribution is [ The result [0.6, 0.3, 0.1] indicates that the proportion of texture defects is 60%, the proportion of structural fracture defects is 10%, and the proportion of local peeling defects is 30%. This result accurately quantifies the defect composition of the image under test and provides a core quantitative basis for subsequent weight optimization.

[0069] In this embodiment, weighted fusion of multi-scale error images is achieved by preset initial weights, and the defect error features of each scale are integrated into a single initial fusion difference map. This avoids feature confusion caused by the separate identification of multiple error images, provides a unified and standardized feature input for defect type and probability distribution identification, and improves the stability of the identification results. By adopting a lightweight identification model, the amount of computation can be reduced while ensuring identification accuracy.

[0070] In one embodiment, defect identification based on the optimized fusion difference map includes: determining a defect threshold based on the defect type and probability distribution; performing threshold segmentation on the optimized fusion difference map based on the determined defect threshold to generate a binary defect mask; and performing defect identification based on the binary defect mask.

[0071] The defect threshold refers to the critical pixel value used to distinguish between defective and normal areas in the optimized fusion difference map. If the value of a pixel in the optimized fusion difference map is higher than the threshold, it is determined to be a defective area; if it is lower than or equal to the threshold, it is determined to be a normal area. The range of the defect threshold is consistent with the range of pixel values ​​in the optimized fusion difference map.

[0072] In this embodiment, based on the scale adaptation characteristics of defect types, the optimized fusion difference map can first be divided into multiple pixel regions such as texture defect feature regions, structural fracture defect feature regions, and local peeling defect feature regions. Then, based on the defect probability distribution, a regionalized defect threshold is set for each region, while a basic threshold t0 and a threshold adjustment coefficient y are introduced. i This enables the quantitative adaptive determination of the threshold. i This represents the threshold adjustment coefficient for the i-th defect type (e.g., i=1,2,3). The defect threshold is positively correlated with the corresponding base threshold and the threshold adjustment coefficient, while the threshold adjustment coefficient is positively correlated with the corresponding defect probability. Negative correlation, meaning that the higher the defect probability, the smaller the adjustment coefficient, and the greater the reduction in the threshold.

[0073] For example, it can be calculated according to the following formula t i = t0×y i = t0×(1- To calculate the defect threshold t of the feature region corresponding to the i-th type of defect. i The defect type with the highest defect probability is used as the defect threshold for the corresponding feature region.

[0074] This formula can ultimately yield a pixel-level defect threshold map t(x,y) of the optimized fusion difference map. This map is the same size as the optimized fusion difference map, and the threshold of each pixel position matches the defect feature area to which it belongs. This achieves the adaptive setting goal of "reducing the threshold of the dominant defect, increasing the threshold of the secondary defect, and making the threshold of the defect-free area close to the base value".

[0075] For the calculated defect threshold, various thresholding methods can be used to adapt to different scenarios: Otsu global threshold: suitable for cases where the difference map histogram has a bimodal distribution; Adaptive threshold: based on local neighborhood mean or Gaussian weighting, suitable for difference maps with uneven lighting; Depth estimation threshold: trains a small convolutional network to predict pixel-level threshold maps. This network takes the difference map as input and outputs the threshold for each pixel, requiring additional labeled data.

[0076] This embodiment calculates the corresponding basic threshold using a corresponding adaptive thresholding method based on different scenarios, and then adjusts the threshold based on the aforementioned threshold adjustment coefficient y. i and defect probability Finally, the defect threshold for the corresponding pixel is calculated. The segmentation rule is as follows:

[0077] in, Let be the pixel value of the binary defect mask at (x, y). To optimize the pixel value at (x,y) in the fusion difference map, is the threshold of the defect threshold map at (x,y).

[0078] This embodiment determines pixel-level defect thresholds based on defect type and probability distribution, ensuring that the threshold setting matches the actual defect features of the image under test. The threshold for dominant defect areas is appropriately lowered to enhance defect feature recognition, while the threshold for secondary defect areas is appropriately raised to suppress background noise interference, thereby improving the accuracy of defect recognition. By dividing the optimized fusion difference map into different defect feature regions and setting personalized thresholds for each region, accurate segmentation of multiple types and mixed defects is achieved, avoiding the problem of insufficient adaptability of a single threshold to different types of defects.

[0079] In one embodiment, feature reconstruction is performed independently for each scale image, and a corresponding reconstructed image is obtained for each scale image. This includes: inputting each scale image into a preset multi-layer reconstruction network model, with each scale image configured with an independent encoder-decoder sub-network; and the corresponding encoder-decoder sub-network performing feature extraction and image reconstruction on the corresponding scale image to obtain a reconstructed image of the corresponding scale.

[0080] In this embodiment, the overall architecture of the multi-layer reconstruction network model consists of encoder-decoder sub-networks that correspond one-to-one with the number of scales. Its core characteristics are scale independence and cascaded optimization. This ensures that the reconstruction accuracy of images at each scale adapts to their own features, while also enabling feature complementarity between scales through cascading. The encoder-decoder sub-network refers to an independent network unit composed of an encoder and a decoder. The encoder is responsible for multi-level feature extraction from the input image, and the decoder is responsible for restoring the extracted features to an image of the same size as the input. It is the basic network unit for feature reconstruction. By configuring a dedicated encoder-decoder sub-network for each scale image, the network parameters and structural details of each sub-network can be flexibly adjusted according to the image features of the corresponding scale without interference, ensuring the reconstruction accuracy of images at different resolution scales.

[0081] Image reconstruction refers to the process by which the decoder, through operations such as upsampling, feature fusion, and convolution, restores the feature maps extracted by the encoder from the feature space to the pixel space, generating an image with the exact same size and number of channels as the input image. The reconstructed image is the output image of the encoder-decoder subnetwork, denoted as... , and the corresponding input scale image I k One-to-one correspondence is the core foundation for subsequent calculation of reconstruction errors and identification of defects. The reconstructed image of the normal region is highly similar to the original image, while the reconstructed image of the abnormal region has obvious pixel deviations.

[0082] Taking the above four scale images as examples, the presupposed multi-layer reconstruction network model is a combination of four independent encoder-decoder sub-networks, denoted as {Net1, Net2, Net3, Net4}, where Net... k The encoder-decoder subnetwork corresponds to the k-th scale, and the structural depth of each subnetwork is appropriately simplified as the scale resolution decreases (e.g., the number of convolutional layers in the subnetwork at scale 4 is reduced by 20% compared to scale 1) to adapt to the feature extraction requirements of low-resolution images and avoid overfitting.

[0083] During the feature reconstruction stage, four scale images that have undergone normalization preprocessing (pixel values ​​normalized to [0,1], and the number of channels unified to 3) are input one-to-one to the corresponding encoder-decoder sub-networks: the first scale image I1 is input to Net1, the second scale image I2 is input to Net2, the third scale image I3 is input to Net3, and the fourth scale image I4 is input to Net4. Each sub-network performs feature extraction and reconstruction operations in parallel, with no parameter sharing and no feature interference, ensuring the independence of multi-scale reconstruction.

[0084] Meanwhile, to improve reconstruction accuracy, each sub-network adopts a cascaded input optimization strategy: the output reconstructed image of the previous scale sub-network is upsampled to the current scale image size through bilinear interpolation and then superimposed with the original input image of the current scale as the actual input of the current scale sub-network. Through cascading, the high-resolution scale features are supplemented to the low-resolution scale, thereby improving the structural reconstruction accuracy of the low-resolution scale.

[0085] In this embodiment, the encoder-decoder subnetworks at each scale perform operations according to the logic of feature extraction-feature transfer-image reconstruction. Taking Net1 as an example, the specific execution process is as follows: Feature extraction: The encoder performs multi-level convolution operations on the input scale image I1, and sequentially extracts feature maps of 64 channels, 128 channels, 256 channels and 512 channels, which correspond to the shallow texture features, mid-level local structure features and deep global morphological features of the image, respectively. Feature transfer: By using skip connections, the feature maps of each level extracted by the encoder are directly transferred to the corresponding layer of the decoder, preserving the detailed features of the image and avoiding the loss of details during the feature extraction process; Image reconstruction: The decoder gradually restores the deep, high-channel-count feature map to its original size through upsampling operations, and simultaneously fuses it with the feature map passed from the encoder. Finally, a 512×512 reconstructed image of the same size as the input is generated through 3×3 convolution and a sigmoid activation function. .

[0086] The remaining sub-networks at other scales are executed using the same logic, adjusting parameters such as the stride of the convolutional layers and the number of feature map channels only according to the scale resolution, ultimately resulting in four reconstructed images that correspond one-to-one with the input scale image. , , , },in 512×512, 256×256, 128×128, The resolution is 64×64, and the pixel positions of all reconstructed images correspond one-to-one with those of the original scale images, providing scale consistency assurance for subsequent error calculations.

[0087] In one embodiment, the encoder in the encoder-decoder subnetwork is built based on the ResNet network. The encoder construction process includes: removing the global average pooling layer and fully connected layer of the ResNet network, and retaining the convolutional layer part; replacing the last 3×3 standard convolution in each residual block group of the ResNet network with a dilated convolution to obtain an encoder adapted to scale image feature extraction.

[0088] Specifically, an encoder can be built based on ResNet-18 as the backbone. First, the classification-related redundant layers of the traditional ResNet-18 are removed. The network structure of the traditional ResNet-18 is: input layer → 7×7 convolutional layer → 3×3 max pooling layer → 4 residual block groups (layer1-layer4) → global average pooling layer → fully connected layer → softmax output layer; the improved encoder structure in this application is: input layer → 7×7 convolutional layer → 3×3 max pooling layer → 4 residual block groups (layer1-layer4) → feature output layer, directly removing the global average pooling layer and fully connected layer, while retaining all convolutional layers and residual connection modules.

[0089] The core purpose of removing redundant layers is to preserve the two-dimensional spatial structure of the feature map: the global average pooling layer and fully connected layer of the traditional ResNet-18 will lose the spatial location information of the feature map and only output a one-dimensional feature vector for classification. However, industrial defect detection requires accurate spatial features (such as the location and shape of the defect). After removal, the encoder output is still a two-dimensional feature map, which preserves the spatial location information of the pixels and provides a foundation for the accurate reconstruction and defect localization of the subsequent decoder.

[0090] Furthermore, the four residual block groups (layer1-layer4) of ResNet-18 are uniformly replaced with dilated convolutions. The specific rule is: the 3×3 standard convolution in the last residual block of each residual block group is replaced with a 3×3 dilated convolution with a dilation rate of 2. The remaining convolutional layers remain unchanged, and the basic structure of each residual block is still "1×1 dimensionality reduction convolution → 3×3 dilated convolution → 1×1 dimensionality increase convolution + residual skip connection".

[0091] Taking layer 1 (the first residual block group) as an example, it contains two residual blocks. The 3×3 standard convolution in the second residual block is replaced with a dilated convolution with a dilation rate of 2. After the replacement, the feature extraction process of this residual block is as follows: input 64-channel feature map → 1×1 convolution to reduce dimensionality to 32 channels → 3×3 dilated convolution (dilation rate 2) to extract features → 1×1 convolution to increase dimensionality to 64 channels → residual concatenation with the original input feature map → output 64-channel feature map. Layers 2-4 are replaced according to the same rules, finally obtaining an encoder for scale-adapted image feature extraction.

[0092] Taking the scale image I1 of a mobile phone glass cover as an example, when the micro-cracks (width < 3 pixels) on the edge of the board are extracted by a standard convolutional encoder, only the local pixel features of the crack can be captured. However, when extracted by a dilated convolutional encoder, its effective receptive field is expanded from 3×3 to 7×7, which can simultaneously capture the local texture features of the micro-crack and the surrounding edge structure features. This allows the decoder to reconstruct the crack area more accurately and improve the residual contrast of the defect area. For macroscopic defects such as structural fractures, dilated convolution can capture a larger range of global structural information and avoid structural distortion during reconstruction.

[0093] In this embodiment, the encoders for all four scales are constructed according to the above rules, and the stride of the convolutional layers is adjusted only according to the scale resolution (e.g., the stride of the max pooling layer of the encoder at scale 4 is changed to 1 to avoid over-downsampling of the feature map), and finally four improved ResNet-18 encoders adapted to the corresponding scale images are obtained.

[0094] This application configures an independent encoder-decoder subnetwork for each scale image, allowing the reconstruction accuracy of each scale to adapt to its own resolution characteristics. The high-resolution scale focuses on feature extraction and reconstruction of minute texture defects, while the low-resolution scale focuses on feature extraction and reconstruction of macroscopic structural defects. This solves the problem that traditional single-network reconstruction cannot take into account defects at multiple scales, achieving full-scale defect feature coverage from micro to macro. By introducing a cascaded input optimization strategy, the reconstruction features of the high-resolution scale are supplemented to the low-resolution scale, achieving feature complementarity between scales, improving the structural reconstruction accuracy of the low-resolution scale, and making the reconstruction residuals of macroscopic defects such as edge breaks and local peeling more significant.

[0095] In one embodiment, the decoder adopts a U-Net-style skip connection structure. The process of reconstructing the encoder output features by the decoder includes: gradually restoring the resolution of the feature map corresponding to the scale image through upsampling operations, and in each upsampling stage, concatenating and fusing the decoder features with the features of the corresponding layer of the encoder; performing convolutional dimensionality reduction and feature extraction on the concatenated and fused features, and finally generating a reconstructed image of the same size as the input scale image.

[0096] The decoder's structure is symmetrically designed with the encoder (e.g., the encoder has 4 residual block groups, and the decoder has 4 upsampling stages). Taking the improved ResNet-18 encoder as an example, its decoder has a U-Net structure with 4 upsampling stages. The specific resolution restoration and feature stitching fusion process is as follows: Upsampling stage 1: The 512-channel, 16×16 feature map output by the encoder (layer4 output) is upsampled to 32×32 through transposed convolution (512→256 channels), and then merged with the 256-channel, 32×32 feature map output by encoder layer3. After merging, the number of channels in the feature map is 256+256=512, and the size is 32×32. Upsampling stage 2: The 512-channel, 32×32 fused feature map is upsampled to 64×64 through transposed convolution (512→128 channels), and then spliced ​​and fused with the 128-channel, 64×64 feature map output from encoder layer 2. After splicing, the number of channels is 128+128=256 and the size is 64×64. Upsampling stage 3: The 256-channel, 64×64 fused feature map is upsampled to 128×128 through transposed convolution (256→64 channels), and then spliced ​​and fused with the 64-channel, 128×128 feature map output by encoder layer 1. After splicing, the number of channels is 64+64=128 and the size is 128×128. Upsampling stage 4: The 128-channel, 128×128 fused feature map is upsampled to 256×256 by transposed convolution (128→32 channels), and then upsampled to 512×512 by bilinear interpolation, restoring the resolution to the original scale 1 image.

[0097] The feature maps passed from the encoder to the decoder need to be adjusted for channel count using a 1×1 convolution to ensure they match the channel count of the feature maps from the corresponding upsampling stage of the decoder before being concatenated and fused. For example, if the encoder layer 3 outputs a 256-channel feature map and the decoder upsampling stage 1 outputs a 256-channel feature map, they can be directly concatenated if the channel counts match. If the channel counts do not match, a 1×1 convolution is used to increase / decrease the dimensionality to match before concatenation to avoid dimensionality conflicts during feature fusion.

[0098] Decoders at other scales are designed according to the above symmetrical structure, adjusting only the number of upsampling stages and the number of channels of transposed convolution based on the scale resolution (e.g., the decoder at scale 4 only needs 2 upsampling stages) to ensure that the decoder at each scale can accurately recover the resolution of the corresponding feature map.

[0099] In each upsampling stage of the decoder, after feature concatenation and fusion are completed, two 3×3 convolution operations are immediately performed to achieve convolutional dimensionality reduction and feature extraction. The specific rule is as follows: the concatenated and fused feature map is halved in the number of channels by the first 3×3 convolution (convolutional dimensionality reduction), and then deep feature extraction is performed by the second 3×3 convolution. The output feature map is used as the input for the next upsampling stage.

[0100] Taking the decoder of scale image I1 as an example, the convolutional dimensionality reduction and feature extraction process at each stage is as follows: Upsampling stage 1: 512-channel feature map after stitching → 3×3 convolution to reduce dimensionality to 256 channels → 3×3 convolution feature extraction → Output 256-channel, 32×32 feature map; Upsampling stage 2: 256-channel feature map after stitching → 3×3 convolution to reduce dimensionality to 128 channels → 3×3 convolution feature extraction → Output 128-channel, 64×64 feature map; Upsampling stage 3: 128-channel feature map after stitching → 3×3 convolution to reduce dimensionality to 64 channels → 3×3 convolution feature extraction → Output 64-channel, 128×128 feature map; After the final upsampling in upsampling stage 4, a 32-channel, 512×512 feature map is obtained. This is followed by a 3×3 convolution to reduce the number of channels to 3 (matching the number of channels in the RGB image). Then, the pixel values ​​are mapped to [0,1] using the Sigmoid activation function (consistent with the pixel range of the input scale image), ultimately generating a 512×512, 3-channel reconstructed image. .

[0101] If the input scale image I1 contains microcracks and edge textures of a mobile phone glass cover, the encoder extracts the texture details of the microcracks and the structural features of the edges, and passes them to the decoder via skip connections. The decoder gradually restores the resolution through upsampling, and simultaneously fuses the detail features and structural features. After convolutional dimensionality reduction and feature extraction, the reconstructed image is generated. In the image, the normal glass cover area highly overlaps with the original image, while the microcrack area shows obvious pixel deviation. This achieves the core objective of "accurate reconstruction of normal areas and high residual in abnormal areas," providing clear error characteristics for subsequent defect detection.

[0102] By removing the global average pooling layer and fully connected layer of ResNet, the two-dimensional spatial structure of the feature map is preserved, providing spatial location information for accurate defect localization and solving the problem of feature loss in traditional classification networks. In addition, the 3×3 standard convolutions of each residual block group are replaced with dilated convolutions with a dilation rate of 2, which expands the effective receptive field from 3×3 to 7×7 without reducing the resolution. This allows the encoder to capture both the local details of small defects and the global structural features of macroscopic defects, improving the richness and comprehensiveness of feature extraction and laying the foundation for accurate reconstruction by the decoder.

[0103] For the decoder, a U-Net-style skip connection + symmetrical upsampling is adopted to directly pass the feature maps of each layer of the encoder to the corresponding layer of the decoder. This preserves the shallow texture details and avoids the loss of details during the decoder upsampling process, making the texture of the reconstructed image clearer and the residual contrast of defects such as tiny cracks and fine scratches higher.

[0104] The decoder and encoder are designed symmetrically. Through the steps of "transposed convolution upsampling + feature splicing and fusion + convolution dimensionality reduction", the resolution of the feature map is gradually restored. At the same time, the detailed features of the encoder and the structural features of the decoder are fused to make the structure of the reconstructed image more complete, avoid structural distortion, and improve the detection accuracy of defects such as structural breaks.

[0105] In one embodiment, the method further includes: training a multi-layer reconstruction network model using a sample image set to obtain a trained multi-layer reconstruction network model, wherein the joint loss function used during training is a weighted sum of the pixel reconstruction loss function, the feature consistency loss function, and the gradient preservation loss function.

[0106] For the aforementioned multi-layer reconstruction network model, taking the detection of defects in mobile phone glass covers as an example, a sample image set is constructed: 10,000 RGB color images of mobile phone glass covers without any defects are collected, covering normal samples from different batches, different shooting angles, and different lighting conditions to avoid model overfitting caused by a single sample; data augmentation is performed on the sample images (e.g., random rotation from 0-90°, random horizontal / vertical flipping, slight brightness adjustment) to expand the sample size to 40,000 images, improving the model's generalization ability; multi-scale downsampling is performed on all sample images to generate a 4-scale image set consistent with the detection stage, which serves as the training input for the model; the sample image set is divided into a training set, a validation set, and a test set in an 8:1:1 ratio. The training set is used for model parameter iteration, the validation set is used for adjusting hyperparameters, and the test set is used to evaluate the model's reconstruction accuracy.

[0107] The model training process is as follows: Initialize all parameters of the multi-layer reconstruction network model (e.g., using Kaiming normal initialization), set the optimizer to Adam, the learning rate to 0.001, the batch size to 16, and the number of training epochs to 100. Input the four-scale sample images from the training set into the model, and train the encoder-decoder subnetworks at each scale in parallel, using an end-to-end training approach, i.e., synchronously iterating the parameters throughout the entire process from the input image to the output reconstructed image. After each epoch of training, evaluate the model's reconstruction accuracy using the validation set (with average pixel error as the evaluation metric). If the validation set accuracy shows no improvement for 10 consecutive epochs, reduce the learning rate (with a decay coefficient of 0.5) and stop training early to avoid overfitting. After training, validate the model's reconstruction effect using the test set.

[0108] During training, the joint loss function used is a weighted sum of the pixel reconstruction loss function, the feature consistency loss function, and the gradient preservation loss function. The pixel reconstruction loss function constrains the model's reconstruction accuracy at the pixel level, calculating the pixel-wise error between the original input image and the reconstructed image, forcing the model to reproduce the texture features of normal samples at the pixel level. The feature consistency loss function constrains the model's reconstruction accuracy at the feature level, calculating the feature error between the original input image and the reconstructed image in the encoder feature space, ensuring semantic feature consistency between the reconstructed image and the original image, and avoiding texture distortion. The gradient preservation loss function constrains the model's reconstruction accuracy at the edge gradient level, calculating the edge gradient error between the original input image and the reconstructed image, ensuring clear edges in the reconstructed image, avoiding blurring, and is crucial for detecting edge defects such as tiny cracks.

[0109] The mathematical expression for the joint loss function can be: L = λ1L rec +λ2L feat +λ3L grad Where L is the joint loss value, L rec For pixel reconstruction loss, L feat For feature consistency loss, L grad For gradient preservation loss, λ1, λ2, and λ3 are weight coefficients, such as λ1=1.0, λ2=0.5, and λ3=0.1. This joint loss function is applicable to sub-networks at all four scales, ensuring the consistency of accuracy in multi-scale reconstruction.

[0110] Regarding pixel reconstruction loss L rec The L1 loss method is used to ensure local reconstruction capability. The formula is as follows:

[0111] in, For the original input image at scale k, For the reconstructed image at scale k, Where L is the L1 norm (sum of absolute errors per pixel), and N is the number of scales for reconstructing the image, for example, 4.

[0112] Feature consistency loss L feat The L1 loss is used to ensure semantic consistency in the feature space. The formula is as follows:

[0113] in, Let N be the feature map output by the l-th residual block group of the k-th scale encoder, where N is the number of scales of the reconstructed image and M is the number of residual block groups, for example, both of which are 4.

[0114] Gradient Preservation Loss L gradThe L1 loss method is used to ensure clear edges in the reconstructed image. The formula is as follows:

[0115] in, The edge gradient of the original input image (calculated using the Sobel operator). To reconstruct the edge gradients of an image, gradient calculation can capture the edge and texture details of the image.

[0116] By employing a joint loss function that combines pixel reconstruction, feature consistency, and gradient preservation, the model training is constrained from three dimensions: pixels, features, and edges. This avoids reconstruction defects caused by a single loss (such as blurred edges and texture distortion), improves the model's reconstruction accuracy for normal regions, and generates significant residuals for the reconstruction of abnormal regions in multiple dimensions. This maximizes the error contrast between normal and abnormal regions, providing clear and reliable error features for subsequent defect identification.

[0117] In one embodiment, taking the aforementioned multi-scale images, which include a first-scale image of the original image to be tested, a second-scale image downsampled by 2x, a third-scale image downsampled by 4x, and a fourth-scale image downsampled by 8x, as an example, the high-resolution scale error image is the first-scale image and the second-scale image. These images retain pixel-level texture details and have the strongest characterization ability for texture defects (such as microcracks, fine scratches, and pinholes, characterized by small size, significant texture features, and obvious pixel-level differences). They can clearly reflect the key features such as the edges and contours of minute defects. The medium-resolution scale error image is the third-scale image. The image, which balances detail preservation and global structure, has the best ability to represent structural fracture defects (such as local edge fractures and product structural damage, characterized by abrupt changes in local morphology and medium-scale structural anomalies), and can accurately capture the structural change patterns of defects. The low-resolution scale error image, also known as the fourth-scale image, loses local details but enhances the global morphology and overall structural features of the image. It has the strongest ability to represent local peeling / long scratch defects (such as large-scale peeling of glass layers and long-distance scratches, characterized by large scale and significant global morphological features), and can effectively distinguish the global structural differences between such defects and normal areas.

[0118] Based on this, the core directions for weight adjustment are defined as follows: 1) If the image to be tested is mainly characterized by texture defects, the direction of weight optimization is to increase the weight of high-resolution scales and appropriately reduce the weight of medium / low-resolution scales, so that the small defect features of high-resolution error images dominate in the fusion process; 2) If the image is mainly characterized by structural fracture defects, the optimization direction is to increase the weight of medium-resolution scales to strengthen the error features of structural anomalies; 3) If the image is mainly characterized by local peeling / long scratch defects, the optimization direction is to increase the weight of low-resolution scales to highlight the differences in defects in the global structure; 4) If the image is characterized by mixed defects, the optimization direction is to adjust the weights of multiple dominant scales simultaneously according to the proportion of each type of defect, and the optimization weights in different defect regions of the error image can be different, so as to further enhance the features of multiple defects simultaneously.

[0119] For error images at these different scales, the following optimization weights can be designed. The calculation formula is as follows:

[0120] in, This represents the initial weights of the image at scale k. The sum of the N initial weights is 1. For example, it can be set to... =0.4, =0.3, =0.2, =0.1; T is the total number of defect types. For example, if T=3, t=1 represents texture defects, t=2 represents structural fracture defects, and t=3 represents local peeling or scratch defects. The sum of the defect probabilities of the three types of defects is 1. The scale-defect fit matrix represents the fit between defect type t and scale k (i.e., whether scale k is the dominant scale for defect t), and takes a value of 0 or 1. When =1, it indicates that scale k is the dominant scale of defect t. When = 0, it indicates that the scale k is not the dominant scale of the defect t; in this embodiment, an adaptation matrix can be set. In this context, rows represent defect type t, and columns represent scale k; for example, the dominant scale for a texture defect with t=1 is k=1 or 2.

[0121] This is the weighting factor. The overall increase in the dominant scale weight is used to control the degree of increase, and can be flexibly adjusted according to industrial scenarios (such as micro-defect detection scenarios). A resolution of 1.8~2.0 is used to enhance high-resolution scale; this is suitable for large structural component inspection scenarios. Take 1.2~1.5). This represents the overall advantage coefficient corresponding to scale k.

[0122] Furthermore, when the probability of a certain type of defect approaches 1, to avoid excessive dilution of weights on other scales, the optimization weights can be slightly adjusted. Specifically, if no defect occurs... Then keep the above calculations. As an optimization weight, if it exists Then, according to the following formula, for Make corrections, and make the corrected version replace , which serves as the final optimization weight. This is the critical probability threshold, for example, 0.9. The correction formula can be:

[0123] Where b is a correction factor, which can be 0.05 or 0.1, for example. Distribute the basic correction weights evenly across N scales to ensure that the sum of the corrected optimization weights is still 1.

[0124] This formula can further improve the rationality of setting optimization weights.

[0125] In one embodiment, a computer-readable storage medium is provided having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the steps in the above method embodiments.

[0126] In one embodiment, an electronic device is also provided, including one or more processors; and a memory storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the steps in the above method embodiments.

[0127] In one embodiment, such as Figure 2 The diagram illustrates the structure of an electronic device used to implement an embodiment of this application. The electronic device includes a central processing unit (CPU) 201, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 202 or a program loaded from a storage portion 208 into a random access memory (RAM) 203. The RAM 203 also stores various programs and data required for the operation of the electronic device. The CPU 201, ROM 202, and RAM 203 are interconnected via a bus 204. An input / output (I / O) interface 205 is also connected to the bus 204.

[0128] The following components are connected to I / O interface 205: an input section 206 including a keyboard, mouse, etc.; an output section 207 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 208 including a hard disk, etc.; and a communication section 209 including a network interface card such as a LAN card, modem, etc. The communication section 209 performs communication processing via a network such as the Internet. Drive 210 is also connected to I / O interface 205 as needed. Removable media 211, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 210 as needed so that computer programs read from them can be installed into storage section 208 as needed.

[0129] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer-readable medium carrying instructions that, in such embodiments, can be downloaded and installed from a network via communication section 209, and / or installed from removable medium 211. When the instructions are executed by central processing unit (CPU) 201, the various method steps described in this application are performed.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0131] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this application and form different embodiments. For example, all the embodiments above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A defect detection method based on feature reconstruction, characterized in that, The method includes: Acquire the image to be tested; The image to be tested is downsampled at multiple scales to form an image at multiple scales containing the image to be tested; Each scale image is reconstructed independently, resulting in a corresponding reconstructed image for each scale image. Calculate the error between the image at each scale and the corresponding reconstructed image to obtain the corresponding error image; Identify the defect type and probability distribution of the image under test; The optimized weights for the error images at each scale are determined based on the defect type and probability distribution. The error images at multiple scales are re-weighted and fused according to the optimized weights to obtain an optimized fusion difference map, and defect identification is performed based on the optimized fusion difference map.

2. The method according to claim 1, characterized in that, The identification of the defect type and probability distribution of the image under test includes: Error images at multiple scales are weighted and fused according to preset initial weights to obtain an initial fusion difference map; Identify the defect types and probability distributions of the fused difference map.

3. The method according to claim 1, characterized in that, The defect identification based on the optimized fusion difference map includes: Based on the defect type and probability distribution, a defect threshold is determined. The optimized fusion difference map is then segmented according to the determined defect threshold to generate a binary defect mask. Defect identification is then performed based on the binary defect mask.

4. The method according to claim 1, characterized in that, The step of independently reconstructing features for each scale of the image, resulting in a corresponding reconstructed image for each scale, includes: Each scale image is input into a pre-defined multi-layer reconstruction network model, with each scale image configured with an independent encoder-decoder sub-network; The corresponding encoder-decoder subnetwork performs feature extraction and image reconstruction on the corresponding scale image to obtain the reconstructed image of the corresponding scale.

5. The method according to claim 4, characterized in that, The encoder in the encoder-decoder subnetwork is built based on the ResNet network. The construction process of the encoder includes: removing the global average pooling layer and fully connected layer of the ResNet network, and retaining the convolutional layer part; replacing the last 3×3 standard convolution in each residual block group of the ResNet network with a dilated convolution to obtain an encoder adapted to scale image feature extraction.

6. The method according to claim 5, characterized in that, The decoder adopts a U-Net-style skip connection structure, and the process of reconstructing the encoder output features by the decoder includes: The resolution of the feature map corresponding to the scale image is gradually restored through upsampling operations, and in each upsampling stage, the decoder features are spliced ​​and fused with the features of the corresponding layer of the encoder. The spliced ​​and fused features are subjected to convolutional dimensionality reduction and feature extraction to finally generate a reconstructed image of the same size as the input image.

7. The method according to claim 4, characterized in that, The method further includes: training a multi-layer reconstruction network model using a sample image set to obtain a trained multi-layer reconstruction network model. The joint loss function used during training is a weighted sum of the pixel reconstruction loss function, the feature consistency loss function, and the gradient preservation loss function.

8. The method according to any one of claims 1 to 7, characterized in that, The defect types include texture defects, structural fracture defects, and local peeling or scratch defects; the determination of optimization weights for the error image at each scale based on the defect types and probability distributions includes: The optimized weight values ​​for error images at high-resolution scales are positively correlated with the probability of texture defects; and / or The optimized weight values ​​for error images at medium resolution scales are positively correlated with the probability of structural fracture defects; and / or The optimized weight values ​​for error images at low resolution scales are positively correlated with the probability of local peeling or scratch defects.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores executable instructions that, when executed by a processor, cause the processor to perform the method as described in any one of claims 1 to 8.

10. An electronic device, characterized in that, include: One or more processors; A memory for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to perform the method as described in any one of claims 1 to 8.

Images