Workpiece surface defect detection method, device and program product based on machine learning

Abstract

The application discloses a workpiece surface defect detection method, device and program product based on machine learning, and the method comprises the steps of: acquiring a workpiece image to be detected; forming an input of a reconstruction network based on the workpiece image to be detected, wherein the reconstruction network comprises a noise adding network, a conditional guidance network and a noise removing network, the noise adding network is connected with the noise removing network, the conditional guidance network and the noise adding network belong to independent branch networks, and the conditional guidance network is connected with the noise removing network; performing a noise adding operation on the workpiece image to be detected through the noise adding network to obtain a noise added latent tensor; performing feature extraction on the workpiece image to be detected through the conditional guidance network to obtain a conditional guidance tensor of anchoring macroscopic topology of the workpiece to be detected; performing an abnormal response erasing operation on the conditional guidance tensor and the noise added latent tensor through the noise removing network to determine a noise removed latent tensor; determining a reconstructed workpiece image based on the noise removed latent tensor; and determining a defect mask based on the workpiece image to be detected and the reconstructed workpiece image.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a method, device and program product for detecting surface defects in workpieces based on machine learning. Background Technology

[0002] With the development of intelligent manufacturing, automated optical inspection (AOI) technology based on computer vision has become a core means to ensure product quality in manufacturing fields such as semiconductors, precision machining, and 3C electronics. Because real defect samples in industrial settings are often scarce, unpredictable, and have extremely high annotation costs, reconstructive unsupervised anomaly detection that relies solely on learning from defect-free samples has become the mainstream technical approach in this field.

[0003] Existing reconstruction-based unsupervised anomaly detection techniques still face significant limitations when applied to complex industrial scenarios involving multiple categories and high resolution. On the one hand, during the denoising and reconstruction process, the lack of strong spatial constraints on the physical topology of heterogeneous workpieces makes the model prone to feature confusion, inducing cross-category semantic drift and nonlinear distortion of local structures. On the other hand, traditional network architectures are prone to causing the loss of minute defect features in deep networks. Summary of the Invention

[0004] To address the existing technical problems, this invention provides a machine learning-based method and system for detecting surface defects in workpieces, which can improve the accuracy of defect detection.

[0005] Firstly, a machine learning-based method for detecting surface defects in workpieces is provided, comprising: acquiring an image of the workpiece to be tested; forming an input to a reconstruction network based on the image of the workpiece to be tested, wherein the reconstruction network includes a noise-adding network, a conditional guidance network, and a denoising network, wherein the noise-adding network and the denoising network are connected, the conditional guidance network and the noise-adding network are independent branch networks, and the conditional guidance network and the denoising network are connected; performing a noise-adding operation on the image of the workpiece to be tested through the noise-adding network to obtain a noise-adding latent tensor; performing feature extraction on the image of the workpiece to be tested through the conditional guidance network to obtain a conditional guidance tensor anchoring the macroscopic topology of the workpiece to be tested; performing an anomaly response erasure operation through the denoising network based on the conditional guidance tensor and the noise-adding latent tensor to determine a denoising latent tensor; determining a reconstructed workpiece image based on the denoising latent tensor; and determining a defect mask image based on the image of the workpiece to be tested and the reconstructed workpiece image.

[0006] In a second aspect, a computer program product is provided, comprising a computer program, characterized in that, when executed by a processor, the computer program implements the workpiece surface defect detection method based on machine learning as described in the first aspect of this application.

[0007] Thirdly, an electronic device is provided, characterized in that it includes a memory and a processor, the memory storing a computer program, which, when executed by the processor, causes the processor to perform the workpiece surface defect detection method based on machine learning as described in the first aspect of this application.

[0008] This application takes the image of the workpiece under test as input. First, it performs data preprocessing and latent space mapping through an autoencoder. Then, under the guidance of multi-scale spatial feature conditions, it completes the denoising of the diffusion model to obtain a denoised latent tensor. The image of the workpiece under test is used as the input of the conditional guided network of the latent space. The conditional guided network extracts the contour features of the image of the workpiece under test to apply effective spatial structure constraints in the reverse denoising stage. Based on the conditional guided tensor and the denoised latent tensor, the denoising network performs anomaly response erasure operation to determine the denoised latent tensor. Finally, the denoised latent tensor is decoded into a high-fidelity defect-free image. Through deep semantic comparison and adaptive statistical threshold segmentation, the final defect mask is output, thereby improving the accuracy of defect detection. Attached Figure Description

[0009] Figure 1 This is an application environment diagram of a machine learning-based workpiece surface defect detection method in one embodiment; Figure 2 This is a flowchart of a machine learning-based workpiece surface defect detection method in one embodiment; Figure 3 This is a schematic diagram of the network structure for reconstructing a network in one embodiment; Figure 4 This is an example diagram of feature fusion in a conditionally guided network in one embodiment; Figure 5 This is an example diagram illustrating the determination of a defect mask image in one embodiment; Figure 6 This is a schematic diagram of a workpiece surface defect detection device based on machine learning in one embodiment; Figure 7 This is a schematic diagram of an electronic device in one embodiment. Detailed Implementation

[0010] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0011] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the specification of this invention is for the purpose of describing particular embodiments only and is not intended to limit the scope of the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0012] In the following description, the expression “some embodiments” refers to a subset of all possible embodiments. However, it should be understood that “some embodiments” can be the same subset or different subsets of all possible embodiments and can be combined with each other without conflict.

[0013] See Figure 1 This is an application environment diagram of a machine learning-based workpiece surface defect detection method in one embodiment. The machine learning-based workpiece surface defect detection system includes an electronic device 10, which can acquire an image of the workpiece to be tested, reconstruct a defect-free reconstructed workpiece image from the image of the workpiece to be tested, and compare the image of the workpiece to be tested with the reconstructed workpiece image to determine a defect mask image.

[0014] With the development of intelligent manufacturing, automated optical inspection (AOI) technology based on computer vision has become a core means to ensure product quality in manufacturing fields such as semiconductors, precision machining, and 3C electronics. Due to the scarcity, unpredictability, and extremely high annotation costs of real defect samples in industrial settings, "reconstruction-based unsupervised anomaly detection," which relies solely on learning from defect-free samples, has become the mainstream technical approach in this field.

[0015] This technology aims to learn the data patterns of normal workpieces using generative models, reconstructing the image to be tested into a defect-free ideal state during inspection, and automatically screening and locating defects by calculating the difference between the original image and the reconstructed image. To adapt to the needs of modern flexible production lines that simultaneously produce hundreds of different workpieces, the industry is using the powerful distribution fitting capabilities of diffusion models to try to achieve "unified anomaly detection for multiple categories," replacing the previously costly "one model per category" independent deployment with a single model.

[0016] Existing reconstruction-based unsupervised anomaly detection techniques still face significant limitations when applied to complex industrial scenarios involving multiple categories and high resolution. On the one hand, during the denoising and reconstruction process, the lack of strong spatial constraints on the physical topology of heterogeneous workpieces makes the model prone to feature confusion, inducing cross-category "semantic drift" and nonlinear distortion of local structures. On the other hand, traditional network architectures are prone to the loss of minute defect features in deep networks, which can easily lead to network training instability or even collapse.

[0017] To address these issues, some methods employ improved strategies that incorporate multi-scale feature fusion. These methods construct multi-level network architectures to extract multi-scale features from industrial images, ranging from local to global, to guide the model's denoising process, thereby improving the accuracy of defect detection.

[0018] In existing technologies, various schemes for industrial surface defect detection and image reconstruction have been proposed. For example, Chinese patent CN119516249B, entitled "Multi-class Defect Detection Method and Device Based on Variational Autoencoder and Denoising Network," discloses a multi-class defect detection method based on a variational autoencoder and a denoising network. This method extracts features from the variational encoder and interacts with the denoising network to guide the denoising network in multi-class reconstruction. While this method can avoid class confusion in multi-class scenarios to some extent, it mainly relies on feature guidance for initial reconstruction. It lacks precise decoupling of deep local textures and targeted hard constraints on complex physical structure boundaries. Therefore, the reconstruction results are still prone to texture confusion between multiple classes and distortion of the original physical structure when dealing with complex edges.

[0019] For example, Chinese patent CN121660981A, entitled "A Method for Detecting Surface Defects of Multi-Category Electronic Components Based on Mamba Architecture," discloses a defect detection method based on the Mamba architecture. This method constructs a multi-scale contextual aggregation branch and explicitly extracts local detail information through channel attention, spatial attention, and parallel multi-scale deep convolution. While this method focuses on enhancing the extraction capability of local defects through attention mechanisms and multi-scale aggregation, it lacks in-depth consideration of cross-layer collaborative fusion between global high-level semantics and extremely shallow high-frequency details. This results in the network easily losing the most original fine-grained features during feature layer-by-layer propagation and dimensionality reduction, and it is not sensitive enough to extremely small micro-cracks, easily leading to the problem of detailed features being submerged by the background.

[0020] For example, Chinese patent CN121746878A, entitled "A Few-Sample Detection Method for Industrial Defects Based on a Conditionally Controlled Diffusion Model," discloses a few-sample detection method based on a conditionally controlled diffusion model. This method generates anomaly scoring maps by calculating cosine similarity on multi-scale feature maps, replacing traditional pixel-level absolute errors. While this method helps mitigate the impact of illumination variations on fixed contrast features, its focus lies in calculating the overall similarity distribution in the feature space. It lacks a dynamic adaptive mask weight allocation mechanism for specific local interferences in real-world industrial scenarios (such as oil stains, rust, and sudden noise). Furthermore, the feature comparison mechanism is prone to failure when faced with highly complex environmental interference, resulting in poor contrast localization capabilities based on fixed feature distributions.

[0021] In summary, while existing industrial defect detection and image reconstruction methods can achieve a certain degree of automated identification and repair, they still have the following shortcomings: First, they focus more on overall feature guidance or multi-scale preliminary aggregation, paying insufficient attention to the decoupling of texture features between categories and the constraints of local physical boundaries, which can easily lead to texture confusion and physical structure distortion during multi-category repair; Second, the network structure fails to effectively preserve global and shallow extremely subtle features without loss, easily losing high-frequency details and being insensitive to small micro-cracks; Third, feature similarity calculation lacks a dynamic adaptive mechanism for complex backgrounds, and traditional comparison approaches are easily affected by environmental interference even when transferred to the feature layer, resulting in poor fixed-value extraction and localization.

[0022] Please see Figure 2 This is a flowchart illustrating a machine learning-based workpiece surface defect detection method according to an embodiment of this application. The machine learning-based workpiece surface defect detection method is applied in electronic devices and includes the following steps: S10. Obtain the image of the workpiece to be tested.

[0023] In this embodiment, the image of the workpiece to be tested represents an image of the workpiece with the defect to be detected. It can be an image acquired by an electronic device using an image sensor, or an image obtained from other devices.

[0024] S11. Based on the image of the workpiece to be tested, the input of the reconstruction network is formed, wherein the reconstruction network includes a noise-adding network, a condition-guided network, and a noise-reducing network.

[0025] In this embodiment, as Figure 3 As shown, Figure 3 This is a schematic diagram of the network structure of a reconstruction network in one embodiment. The reconstruction network includes an encoder, a noise-adding network connected to the encoder, a denoising network connected to the noise-adding network, and a decoder connected to the denoising network. The conditional guidance network and the noise-adding network are independent branch networks. The encoder is used to compress and map the high-resolution image of the workpiece to a low-dimensional latent space to significantly reduce computational overhead, and outputs the original latent tensor compressed by a preset downsampling factor. The original latent tensor As input to a noisy network.

[0026] S12. Using a noise-adding network, perform a noise-adding operation on the image of the workpiece to be tested to obtain a noise-adding latent tensor.

[0027] In this embodiment, during the forward diffusion stage, the total diffusion time step T is preset. The system randomly samples the current time step t from a uniform distribution and generates standard Gaussian noise. By employing reparameterization techniques, the noisy latent tensor at the current time step t is computed in one step, and its expression is: in, This represents the cumulative product variance parameter calculated based on the scheduling strategy. This process gradually destroys the workpiece structure information in the latent space, providing an initial state for inverse denoising.

[0028] S13. Through the conditional guided network, feature extraction is performed on the image of the workpiece to be tested to obtain the conditional guided tensor of the macroscopic topology anchored to the workpiece to be tested.

[0029] In this embodiment, the image of the workpiece under test is used as the input to a conditional guided network. This network extracts features from the global macroscopic structure to the local microscopic texture, resulting in conditional guided tensors at different scales. During the spatial conditional feature extraction stage, the conditional guided network, independent of the denoising backbone, extracts high-dimensional hidden layer features from the workpiece image and generates multi-scale conditional guided tensors. These conditional guided tensors include tensors at multiple different scales.

[0030] S14. Based on the condition-guided tensor and the noisy potential tensor, perform anomaly response erasure operation through a denoising network to determine the denoising potential tensor.

[0031] In this embodiment, at each time step t of the diffusion denoising process, the conditional guidance network injects multi-scale conditional guidance tensors into the intermediate hidden states of the corresponding layers of the inverse upper-level denoising network. The denoising network can be a U-Net-based network. The noisy latent tensor serves as the input to the denoising network, and the conditional guidance tensor injects multi-scale conditional guidance tensors into the intermediate layers of the denoising network. In this way, the denoising network is forced to perform denoising within the geometric framework anchored by the conditional guidance network, thus locking the spatial topology of multiple workpiece categories.

[0032] S15. Based on the denoising latent tensor, determine the reconstructed workpiece image.

[0033] In this embodiment, during the pixel space decoding stage, the denoised latent tensor is fed into the decoder of the spatial autoencoder. The decoder uses multiple upsampling residual blocks to inversely map the low-dimensional features back to the high-dimensional RGB pixel space, outputting a defect-free, normally reconstructed image that is strictly consistent with the spatial scale of the image of the workpiece under test, i.e., the reconstructed workpiece image.

[0034] S16. Based on the image of the workpiece to be tested and the reconstructed image of the workpiece, determine the defect mask image.

[0035] In this embodiment, the defect mask map indicates the location of defects in the working image under test.

[0036] In the above embodiments, the method takes the image of the workpiece to be tested as input. First, it performs data preprocessing and latent space mapping through an autoencoder. Then, under the guidance of multi-scale spatial feature conditions, it completes the denoising of the diffusion model to obtain a denoised latent tensor. The image of the workpiece to be tested is used as the input of the conditional guidance network of the latent space. The conditional guidance network extracts the contour features of the image of the workpiece to be tested to apply effective spatial structure constraints in the reverse denoising stage. Based on the conditional guidance tensor and the denoised latent tensor, the denoising network performs anomaly response erasure operation to determine the denoised latent tensor. Finally, the denoised latent tensor is decoded into a high-fidelity defect-free image, and the final defect mask is output through deep semantic comparison and adaptive statistical threshold segmentation, thereby improving the accuracy of defect detection.

[0037] In some embodiments, the conditional guidance network includes a convolutional subnetwork, a first coding subnetwork, and a first decoding subnetwork cascaded with the first coding subnetwork, and the denoising network includes a second coding subnetwork, wherein the output of the first downsampling layer in the second coding subnetwork is connected to the input of the first coding subnetwork. By using a conditional guided network, feature extraction is performed on the image of the workpiece under test, resulting in a conditional guided tensor that anchors the macroscopic topology of the workpiece under test, including: By using a convolutional subnetwork, a convolution operation is performed on the image of the workpiece under test to obtain a convolution tensor; Obtain the upsampled tensor of the output of the first downsampled layer in the second coding subnetwork; The convolutional tensor and the upsampled tensor are concatenated to obtain the input of the first encoding sub-network. Feature extraction is performed through the first encoding sub-network and the first decoding sub-network to obtain the conditional guiding tensor of the macroscopic topology of the workpiece to be measured.

[0038] In this embodiment, the tensor obtained by concatenating the convolutional tensor and the upsampled tensor along the channel dimension is used as the input to the first encoding sub-network. The second encoding sub-network includes multiple cascaded downsampling layers. Since the input to the second encoding sub-network is a noisy latent tensor, the upsampled tensor output from the first downsampling layer is the contaminated feature tensor. After being concatenated with the convolutional tensor obtained without noise processing, when input to the first encoding sub-network, the first encoding sub-network must upsample the random perturbations of the tensor while aligning with the stable responses in the upsampled tensor. This is equivalent to forcing the first encoding sub-network to perform denoising contrastive learning, causing its weights to tend towards extracting robust deep representations that are insensitive to noise. The final effect is that when the first encoding sub-network faces real defects, such as minor scratches, it will not produce false alarms due to lighting jitter or sensor thermal noise.

[0039] In some embodiments, a feature fusion network connects the first encoding subnetwork and the second encoding subnetwork. Feature extraction is performed through the first encoding subnetwork and the first decoding subnetwork to obtain the conditional guided tensor of the macroscopic topology of the workpiece to be measured, including: For the first decoding subnetwork, the... upsampling layer, where ,in Given the total number of upsampling layers, obtain the first encoding subnetwork. The downsampled feature tensor corresponding to the upsampled layer, where the first... The scale of the downsampled feature tensor corresponding to the upsampled layer is greater than or equal to the first... The scale of the upsampled feature tensor output by the upsampled layer; Through feature fusion networks, the first The downsampled feature tensors corresponding to the upsampled layers are fused to obtain the fused downsampled feature tensor; Based on the The output of the upsampling layer and the fused downsampling feature tensor are used to determine the first... The input of the upsampling layer, the output of the first layer The conditional guiding tensor corresponding to the upsampling layer.

[0040] In this embodiment, the first encoding subnetwork includes multiple cascaded downsampling layers. The first decoding subnetwork includes multiple cascaded upsampling layers.

[0041] No. The scale of the downsampled feature tensor corresponding to the upsampled layer is greater than or equal to the first... The scale of the upsampled feature tensor output by the upsampled layer, then relative to the first... Regarding the upsampled feature tensor output by the upsampled layer, the first... The downsampled feature tensors corresponding to the upsampled layers belong to shallow features, and the spatial resolution of each downsampled feature tensor is greater than that of the first layer. Spatial resolution of the upsampled feature tensor output by the upsampled layer.

[0042] Specifically, no. upsampling layer input The formula is as follows: in, This represents a non-linear activation function. This represents a group normalization operation that forces the division into a fixed number of groups. Indicates the output of the first... The set of layer indices for the downsampling layer corresponding to the downsampling feature tensor of the upsampling layer. Indicates the first The downsampled feature tensor output by the downsampled layer. Indicates the first The output of the upsampling layer. Indicates the first Input to the upsampling layer. This represents a convolution based on a zero-initialized weight matrix, which is an all-zero matrix in the first iteration of training. As training progresses, the zero-initialized weight matrix is ​​gradually updated to have non-zero elements.

[0043] This feature fusion network intercepts shallow feature map groups for fusion injection into medium-depth feature map groups. The deep feature fusion network intercepts deep feature map groups for fusion injection into the deepest feature map groups. The feature fusion network constructs independent mapping sequences for each intercepted downsampled feature tensor. These sequences sequentially include downsampling convolutional layers encapsulated by zero-initialized weight matrices, group normalization layers (GroupNorm), and SiLU activation layers. Specifically, the downsampling convolutional layers downsample the spatial dimensions of the downsampled feature tensors to ensure that the original latent space data distribution of the pre-trained network is not disrupted during the initial training phase; the group normalization layers forcibly divide channels into a fixed number of groups, independently calculating the mean and variance within each group to eliminate statistical estimation bias caused by mini-batch training; and the SiLU activation layers utilize their smooth, non-monotonic properties near zero to preserve weak negative gradients.

[0044] like Figure 4 As shown, the 32×32 features on the right are processed through convolution (conv), group normalization (GN), and activation function (SiLU), and then added to the 16×16 features on the right. That is, the 32×32 and 16×16 features connected by the corresponding green lines are fused together and added to the input of the 16×16 layer on the left through the SiLU activation function. The input of the 16×16 layer on the left also includes the output of the 8×8 layer on the left. The other layers on the left undergo similar processing. Figure 4 The 4×4 level on the left is similar, which is to merge the 4×4 features and 8×8 features connected by the corresponding blue lines.

[0045] The first encoding subnetwork in the conditional guidance network contains multiple downsampling stages, sequentially transforming the input image into multi-scale feature tensors of different spatial sizes (e.g., 32×32, 16×16, 8×8, 4×4) to construct a complete feature pyramid from global macrostructure to local microtexture. The deepest features output by the first encoding subnetwork are then passed to the first decoding subnetwork. The first decoding subnetwork contains corresponding upsampling stages (e.g., 4×4, 8×8, 16×16) to progressively restore the spatial resolution of the feature maps. Specifically, the network introduces a feature fusion network to achieve cross-layer guidance of multi-scale features. This feature fusion network extracts conditional features of specific resolutions from the first encoding subnetwork, fuses them, and precisely injects the processed multi-scale conditional guidance tensor into the corresponding layer of the decoder.

[0046] In the above embodiments, the hierarchical spatial feature fusion network decouples multi-scale features into parallel sub-paths, which can take into account both the detection response to minute anomalies and macroscopic structural anomalies. The feature fusion network internally uses a combination of group normalization (GN) and SiLU activation function, which reduces the dependence of traditional batch normalization on batch size and can still maintain the stability of feature distribution under smaller training batches, which helps to suppress gradient collapse.

[0047] In some embodiments, the denoising network further includes a second decoding subnetwork cascaded with the second coding subnetwork; the target upsampling layer in the second decoding subnetwork is skipped to the first decoding subnetwork. Based on the conditional guidance tensor and the noisy potential tensor, an anomaly response erasure operation is performed through a denoising network to determine the denoising potential tensor, which includes: Based on the noisy latent tensor, the input of the second encoding sub-network is formed, and the operation is performed layer by layer. The output of the second encoding sub-network is then passed to the second decoding sub-network. Based on the condition-guided tensor, the denoising potential tensor is determined through the second decoding sub-network.

[0048] In this embodiment, as Figure 3 As shown, the noisy latent tensor serves as the input to the second coding sub-network. After layer-by-layer execution, the output of the last layer of the second coding sub-network serves as the input to the first layer of the second decoding sub-network. The second decoding sub-network includes one or more upsampling layers. Among these upsampling layers, there are one or more target upsampling layers that have skip connections with the first decoding sub-network. The target upsampling layers in the second decoding sub-network have skip connections with layers of the same scale in the first decoding sub-network. For example, a 16×16 upsampling layer in the second decoding sub-network is a target upsampling layer and has a skip connection with a 16×16 upsampling layer in the first decoding sub-network.

[0049] Optionally, based on the conditionally guided tensor, the denoising potential tensor is determined through the second decoding sub-network, including: For the target upsampling layer, obtain the target conditional guidance tensor of the layer output that is skipped to the target upsampling layer in the first decoding subnetwork; Obtain the output of the layer preceding the target upsampling layer in the second decoding subnetwork; Based on the output of the layer above the target upsampling layer and the target condition guiding tensor, the output of the target upsampling layer is determined until the last layer of the second decoding sub-network is executed, and the denoising potential tensor is obtained.

[0050] In this embodiment, the target upsampling layer refers to the upsampling layer in the second decoding subnetwork that has skip connections. The target conditional guidance tensor refers to the conditional guidance tensor output by the layer in the first decoding subnetwork that has skip connections with the target upsampling layer.

[0051] At each time step t in the denoising process, the smoothed multi-scale conditional tensor is fused element-wise with the corresponding layer of the inverse denoising U-Net backbone encoder. For a target upsampling layer, the input of the target upsampling layer... ,in This represents the output of the layer preceding the target upsampling layer. This represents the target conditional guidance tensor output by a layer that has a skip connection with the upsampled layer of the target. This represents convolution based on a zero-initialized weight matrix.

[0052] In the above embodiments, the latent space conditional guidance network extracts the contour features of the original image and makes skip connections with the second decoding sub-network in the denoising network. This applies effective spatial structure constraints in the reverse denoising stage, which to some extent reduces the semantic drift or macroscopic geometric distortion of the diffusion model during multi-class mixed training, thereby improving the fidelity of the defect-free reconstructed image.

[0053] In some embodiments, determining the defect mask image based on the image of the workpiece under test and the reconstructed workpiece image includes: The image of the workpiece to be tested is input into the feature extraction network to obtain the first feature map at each scale. The reconstructed workpiece image is input into the feature extraction network to obtain the second feature map at each scale. Based on the first feature map and the second feature map at each scale, the difference map at each scale is determined. Stack the difference maps at different scales to obtain a smoothed outlier map; Determine the defect mask image based on the smoothed anomaly map.

[0054] In this embodiment, the feature extraction network can be a pre-trained deep convolutional feature extraction network. The feature extraction network is used to extract the perceptual semantic feature maps of the workpiece image to be tested and the reconstructed workpiece image at the mid-to-deep level, calculate the cosine similarity distance between the two to generate difference maps at various scales, transform the difference maps at various scales to the same size, and stitch them together in the channel dimension to obtain an initial anomaly response map. After smoothing by a two-dimensional Gaussian low-pass filter, a smoothed anomaly map is output.

[0055] Optionally, determining the defect mask map based on the smoothed anomaly map includes: Based on the pixel values ​​of each pixel in the smoothed anomaly map, the pixel mean and pixel standard deviation are calculated. The segmentation threshold is determined based on the pixel mean and pixel standard deviation. Based on the segmentation threshold, the smooth anomaly map is binarized to obtain the defect mask map.

[0056] After range normalization, the smoothed anomaly map is used to generate a pixel-level binary defect mask map using an adaptive segmentation strategy based on global statistical characteristics. The adaptive binarization segmentation threshold is then applied. The calculation formula is: in, Represents the pixel mean. This represents the standard deviation of pixels. This represents the adjustable sensitivity coefficient used to balance the false positive and false negative rates. The system performs dynamic decision-making on the normalized anomaly map; pixel values ​​exceeding a threshold are considered... The connected regions are then identified as defective regions.

[0057] For example, Figure 5 This is an example diagram for determining a defect mask image in one embodiment. This represents the image of the workpiece to be tested. This indicates the reconstruction of the workpiece image. , , These represent different scales. , , They represent , , Differences at different scales.

[0058] By utilizing the mid-to-deep perceptual feature maps from a pre-trained feature extraction network for cosine similarity comparison, environmental noise interference caused by traditional direct pixel subtraction is mitigated, making the system more robust to changes in illumination and minute mechanical displacements. Combined with a dynamic adaptive threshold segmentation method based on mean and standard deviation, the model can adaptively adjust the decision threshold for different product backgrounds, reducing background artifacts and false alarms, and contributing to improved generation accuracy and localization accuracy of the final pixel-level binary defect mask.

[0059] In some embodiments, the method further includes: Obtain the training dataset. Each training sample in the training dataset includes a workpiece sample image and a corresponding defect-free workpiece label image. Based on the training dataset, an iterative method is used to train and reconstruct the network.

[0060] During model training, the system employs an asymmetric training strategy of freezing the backbone and fine-tuning the branches. The original parameters of the encoder in the backbone network of the diffusion model are frozen; only the backpropagation gradients are used to update the convolutional layers and their corresponding zero-initialized parameters in the conditional guided network, achieving adaptive release of conditional constraints. During training, the conditional guided network operates independently of the denoising backbone network, employing the same asymmetric training strategy: freezing the original encoder parameters of the backbone network and updating the parameters of the conditional guided network only using backpropagation gradients.

[0061] For the training dataset, the input industrial images come from different sources and have varying original resolutions. All images are uniformly standardized and scaled to a fixed preset size to ensure spatial consistency of the model's input tensor dimensions. In multi-class mixed training scenarios, the names of each industrial product category are strictly mapped to integer indices, and category feature vectors are generated using a category embedding algorithm.

[0062] This invention accurately locks the spatial topology of multiple types of workpieces by using a conditional guided network without destroying the prior knowledge of the pre-trained diffusion model. By embedding a feature fusion network and an adaptive threshold segmentation mechanism, it effectively overcomes the feature loss problem under extremely small batch training in industrial applications and the localization generalization problem under high contrast span, thus achieving highly robust and high-fidelity unified anomaly detection of multiple types of industrial workpieces.

[0063] During the model inference stage, only the standardized image of the workpiece to be inspected needs to be input into the system. The system automatically completes latent mapping, feature injection, inverse denoising, and decoding reconstruction, and directly generates a high-precision defect mask based on an adaptive statistical threshold, without the need for manually setting a fixed threshold.

[0064] This embodiment reduces the computational requirements of the model and improves structural stability to a certain extent by co-optimizing a pre-trained spatial autoencoder and a zero-initialization conditionally guided network. To address the feature loss problem caused by small-batch training in industrial production, the designed feature fusion network and deep semantic extraction mechanism provide certain compensation and coping solutions. When dealing with multi-class workpiece detection tasks with high contrast spans, this method demonstrates stronger environmental adaptability and positioning accuracy compared to traditional hard thresholding and pure pixel-level comparison methods, showing good potential for practical application.

[0065] In another aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements a machine learning-based workpiece surface defect detection method according to any embodiment of this application.

[0066] Among them, in the computer program product, the optional implementation form of the program module architecture of the computer program that implements each step of the target recognition method can be a workpiece surface defect detection device based on machine learning.

[0067] Please see Figure 6 This application provides a machine learning-based workpiece surface defect detection device, comprising: an acquisition module 61 for acquiring an image of a workpiece to be tested; a reconstruction module 62 for forming an input to a reconstruction network based on the image of the workpiece to be tested, wherein the reconstruction network includes a noise-adding network, a conditional guidance network, and a denoising network, wherein the noise-adding network and the denoising network are connected, the conditional guidance network and the noise-adding network are independent branch networks, and the conditional guidance network and the denoising network are connected; the reconstruction module 62 is further configured to perform a noise-adding operation on the image of the workpiece to be tested through the noise-adding network to obtain a noise-adding latent tensor; the reconstruction module 62 is further configured to perform feature extraction on the image of the workpiece to be tested through the conditional guidance network to obtain a conditional guidance tensor anchoring the macroscopic topology of the workpiece to be tested; the reconstruction module 62 is further configured to perform an anomaly response erasure operation through the denoising network based on the conditional guidance tensor and the noise-adding latent tensor to determine a denoising latent tensor; the reconstruction module 62 is further configured to determine a reconstructed workpiece image based on the denoising latent tensor; and a segmentation module 63 for determining a defect mask image based on the image of the workpiece to be tested and the reconstructed workpiece image.

[0068] Please see Figure 7In another aspect of this application, an electronic device 10 is also provided, including a memory 3011 and a processor 3012. The memory 3011 stores a computer program. When the computer program is executed by the processor, the processor 3012 performs the steps of the workpiece surface defect detection method based on machine learning provided in any of the above embodiments of this application. The electronic device 10 is (e.g., a desktop computer, laptop computer, tablet computer, handheld computer, smart speaker, server, etc.), a mobile phone (e.g., a smartphone, cordless phone, etc.), a wearable device (e.g., a pair of smart glasses or a smartwatch), or a similar device. The transmitting end and the receiving end are different electronic devices, but both the transmitting end and the receiving end include the memory 3011 and the processor 3012 in the electronic device 10, and perform the steps of the workpiece surface defect detection method based on machine learning provided in any of the above embodiments of this application.

[0069] The processor 3012 is the control center, connecting various parts of the electronic device via various interfaces and lines. It executes software programs and / or modules stored in the memory 3011, and calls data stored in the memory 3011 to perform various functions and process data. Optionally, the processor 3012 may include one or more processing cores; preferably, the processor 3012 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user page, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 3012.

[0070] The memory 3011 can be used to store software programs and modules. The processor 3012 executes various functional applications and data processing by running the software programs and modules stored in the memory 3011. The memory 3011 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, application programs required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 3011 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 3011 may also include a memory controller to provide the processor 3012 with access to the memory 3011.

[0071] In another aspect, this application also provides a storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the steps of the workpiece surface defect detection method based on machine learning provided in any of the above embodiments of this application.

[0072] Those skilled in the art will understand that all or part of the processes in the methods provided in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0073] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. The scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for detecting surface defects in workpieces based on machine learning, characterized in that, include: Acquire an image of the workpiece to be tested; Based on the image of the workpiece to be tested, the input of the reconstruction network is formed, wherein the reconstruction network includes a noise-adding network, a conditional guidance network and a denoising network, wherein the noise-adding network is connected to the denoising network, the conditional guidance network and the noise-adding network are independent branch networks, and the conditional guidance network is connected to the denoising network. The noise-adding network is used to perform a noise-adding operation on the image of the workpiece under test to obtain a noise-adding latent tensor. Through the conditional guidance network, feature extraction is performed on the image of the workpiece to be tested to obtain a conditional guidance tensor that anchors the macroscopic topology of the workpiece to be tested. Based on the condition-guided tensor and the noisy potential tensor, the anomaly response erasure operation is performed through the denoising network to determine the denoising potential tensor; Based on the denoising latent tensor, the reconstructed workpiece image is determined; Based on the image of the workpiece to be tested and the reconstructed image of the workpiece, a defect mask image is determined.

2. The workpiece surface defect detection method based on machine learning as described in claim 1, characterized in that, The conditional guidance network includes a convolutional subnetwork, a first coding subnetwork, and a first decoding subnetwork cascaded with the first coding subnetwork. The denoising network includes a second coding subnetwork, wherein the output of the first downsampling layer in the second coding subnetwork is connected to the input of the first coding subnetwork. The step of performing feature extraction on the image of the workpiece under test through the conditional guided network to obtain a conditional guided tensor anchoring the macroscopic topology of the workpiece under test includes: A convolution operation is performed on the image of the workpiece under test through a convolutional subnetwork to obtain a convolution tensor; Obtain the upsampled tensor of the output of the first downsampled layer in the second coding subnetwork; The convolutional tensor and the upsampled tensor are concatenated to obtain the input of the first encoding subnetwork. Feature extraction is performed through the first encoding subnetwork and the first decoding subnetwork to obtain the conditional guiding tensor that anchors the macroscopic topology of the workpiece under test.

3. The workpiece surface defect detection method based on machine learning as described in claim 2, characterized in that, A feature fusion network connects the first encoding subnetwork and the second encoding subnetwork. The step of performing feature extraction through the first encoding subnetwork and the first decoding subnetwork to obtain a conditional guided tensor anchoring the macroscopic topology of the workpiece under test includes: For the first decoding subnetwork, the first upsampling layer, where ,in The total number of upsampling layers is used to obtain the first encoding sub-network. The downsampled feature tensor corresponding to the upsampled layer, where the first... The scale of the downsampled feature tensor corresponding to the upsampled layer is greater than or equal to the first... The scale of the upsampled feature tensor output by the upsampled layer; Through the feature fusion network, the first... The downsampled feature tensors corresponding to the upsampled layers are fused to obtain the fused downsampled feature tensor; Based on the The output of the upsampling layer and the fused downsampling feature tensor determine the first... The input of the upsampling layer, the output of the first layer The conditional guiding tensor corresponding to the upsampling layer.

4. The workpiece surface defect detection method based on machine learning as described in claim 2, characterized in that, The denoising network further includes a second decoding subnetwork cascaded with the second encoding subnetwork; the target upsampling layer in the second decoding subnetwork is skipped to the first decoding subnetwork. The process of determining the denoised potential tensor by performing anomaly response erasure operation through the denoising network based on the condition-guided tensor and the noisy potential tensor includes: Based on the noisy latent tensor, the input of the second encoding sub-network is formed, and the network is executed layer by layer. The output of the second encoding sub-network is then passed to the second decoding sub-network. Based on the condition-guided tensor, the denoising potential tensor is determined through the second decoding sub-network.

5. The workpiece surface defect detection method based on machine learning as described in claim 4, characterized in that, The step of determining the denoising potential tensor based on the condition-guided tensor through the second decoding sub-network includes: For the target upsampling layer, obtain the target conditional guidance tensor of the layer output that is skipped to the target upsampling layer in the first decoding subnetwork; Obtain the output of the layer preceding the target upsampling layer in the second decoding sub-network; Based on the output of the layer preceding the target upsampling layer and the target conditional guidance tensor, the output of the target upsampling layer is determined until the last layer of the second decoding sub-network is reached, thus obtaining the denoising potential tensor.

6. The workpiece surface defect detection method based on machine learning as described in claim 1, characterized in that, The step of determining the defect mask image based on the image of the workpiece to be tested and the reconstructed workpiece image includes: The image of the workpiece to be tested is input into the feature extraction network to obtain the first feature map at each scale, and the reconstructed workpiece image is input into the feature extraction network to obtain the second feature map at each scale. Based on the first feature map and the second feature map at each scale, the difference map at each scale is determined. Stack the difference maps at different scales to obtain a smoothed outlier map; The defect mask map is determined based on the smoothed anomaly map.

7. The workpiece surface defect detection method based on machine learning as described in claim 6, characterized in that, Determining the defect mask image based on the smoothed anomaly image includes: Based on the pixel values ​​of each pixel in the smoothed anomaly map, the pixel mean and pixel standard deviation are calculated. The segmentation threshold is determined based on the mean pixel value and the standard deviation pixel value. Based on the segmentation threshold, the smooth anomaly map is binarized to obtain the defect mask map.

8. The workpiece surface defect detection method based on machine learning as described in claim 1, characterized in that, The method further includes: Obtain a training dataset, wherein each training sample in the training dataset includes a workpiece sample image and a corresponding defect-free workpiece label image; The reconstruction network is trained using an iterative method based on the training dataset.

9. An electronic device, characterized in that, The device includes a memory and a processor, the memory storing a computer program that, when executed by the processor, causes the processor to perform the machine learning-based workpiece surface defect detection method as described in any one of claims 1 to 8.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the workpiece surface defect detection method based on machine learning as described in any one of claims 1 to 8.

Images