A model training and micro-defect detection method for industrial surface micro-defect detection

Abstract

The application relates to a model training and micro-defect detection method for industrial surface micro-defect detection, and belongs to the technical field of computer vision. The application trains an anti-fact expansion model through a first sample training set to generate anti-fact defect sample images, and then obtains a second sample training set. The application obtains the pixel-level defect label confidence of each sample image in the second sample training set through a double-teacher model, trains a student defect detection model according to the second sample training set at this time, and uses the student defect detection model for industrial surface micro-defect detection. Through anti-fact defect sample generation and double-teacher consistency checking, on the basis of reasonable generation of local defects in a normal background, the reliability of defect labels is checked by mutual checking of multiple teacher models, and a more reliable micro-defect detection model is trained. In inference, the student defect detection model is used for micro-defect detection, and the uncertainty is inhibited through detection of the uncertainty, so that the recall rate of micro-defect detection under a complex texture background is improved, and the false alarm rate is reduced.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, specifically to a model training and micro-defect detection method for industrial surface micro-defect detection. Background Technology

[0002] Traditional industrial surface defect detection methods typically rely on threshold segmentation, edge detection, template matching, frequency domain filtering, or manually designed features. However, these methods have high requirements for illumination stability, product posture consistency, and background texture regularity. When the workpiece surface has minute defects such as brush marks, reflections, oil stains, embossed marks, natural textures, or micro-scratches, it is prone to false positives or false negatives.

[0003] While deep learning-based defect detection methods can learn defect features, industrial defects exhibit significant small sample sizes and long tails. Many severe defects have low occurrence probabilities, long acquisition cycles, and defect morphology is greatly affected by materials, processes, lighting, and shooting angles, making it difficult for real-world labeled data to cover all working conditions. Training models based on a small number of defect samples is prone to overfitting; training models based solely on normal samples, on the other hand, can easily misclassify normal texture changes as defects.

[0004] Although defect samples can be expanded using generative models, these models are prone to altering the background texture of products and generating defects that do not conform to physical laws, or generating pseudo-defects that are inconsistent with the real production scenario when generating images. Visual basic models that rely on massive amounts of general images for pre-training have strong segmentation capabilities, but when applied to industrial micro-defect detection scenarios, they are still prone to mistaking textures, reflections, edges, or shadows for defects.

[0005] Therefore, existing methods are easily affected by factors such as the inherent texture of the workpiece, reflection, and shadow, resulting in false detections or missed detections. At the same time, due to the small sample size and long tail characteristics of defects, the generalization ability of the model is insufficient, making it difficult to achieve accurate detection of micro-defects on industrial surfaces. Summary of the Invention

[0006] The main technical problem this invention addresses is how to achieve accurate detection of micro-defects on industrial surfaces.

[0007] According to the first aspect, one embodiment provides a model training method for detecting micro-defects on industrial surfaces, comprising:

[0008] Obtain the first training set;

[0009] The preset diffusion model is trained based on the first sample training set to obtain a counterfactual diffusion model. The counterfactual diffusion model is used to generate counterfactual defect sample images based on normal sample images in the first sample training set. Each pixel unit in the counterfactual defect sample image corresponds to a defect judgment label, and the defect judgment label is used to characterize whether the pixel unit belongs to a defect.

[0010] A second sample training set is obtained based on the first sample training set and all counterfactual defect sample images;

[0011] The pixel-level defect label confidence score of each sample image in the second sample training set is obtained through a dual-teacher model, which includes a first teacher model and a second teacher model. For any sample image in the second sample training set: the first teacher model is used to perform defect semantic recognition on the sample image to obtain a first defect probability map corresponding to the sample image; the second teacher model is used to perform texture recognition on the sample image to obtain a second defect probability map corresponding to the sample image; consistency fusion is performed based on the first defect probability map and the second defect probability map to obtain the defect label confidence score of each pixel unit in the sample image; at this time, each pixel unit of each sample image in the second sample training set corresponds to a defect judgment label and a defect label confidence score.

[0012] A preset student model is trained based on the second sample training set to obtain a student defect detection model, which is used for detecting micro-defects on industrial surfaces.

[0013] In one embodiment, training a preset diffusion model based on the first sample training set to obtain a counterfactual diffusion model includes:

[0014] For any normal sample image in the first sample training set: the normal sample image and its corresponding preset defect type, preset defect location mask and preset defect intensity parameters are input into the diffusion model to obtain the predicted noise corresponding to the normal sample image; the defect location mask is used to determine the generation range of the defect;

[0015] Based on the predicted noise corresponding to the normal sample image, a counterfactual defect sample image of the normal sample image and a defect determination label for each pixel unit in the counterfactual defect sample image are obtained.

[0016] The main loss function is calculated based on the actual noise corresponding to each normal sample image and the predicted noise.

[0017] Calculate the background consistency constraint loss based on the defect location mask corresponding to the normal sample image and the counterfactual defect sample image;

[0018] The defect morphology constraint loss is calculated based on the counterfactual defect sample image. The defect morphology constraint loss is used to ensure the physical shape of the defect region in the counterfactual defect sample image. The defect morphology constraint loss includes continuity constraint, curvature constraint and size constraint. The continuity constraint is used to constrain the continuity of the defect in the counterfactual defect sample image. The curvature constraint is used to constrain the curvature smoothness of the defect in the counterfactual defect sample image. The size constraint is used to constrain the size range of the defect in the counterfactual defect sample image.

[0019] The counterfactual total loss function is obtained based on the main loss function, the background consistency constraint loss, and the defect morphology constraint loss.

[0020] The diffusion model is trained using the counterfactual total loss function, and the trained diffusion model is used as the counterfactual diffusion model.

[0021] In one embodiment, the second teacher model is used to perform texture recognition on the sample image to obtain a second defect probability map corresponding to the sample image, including:

[0022] A pre-trained texture reconstruction model is obtained, which is obtained by training a preset model on the first sample training set to learn the normal texture distribution.

[0023] The sample image is input into the texture reconstruction model to obtain the reconstructed image corresponding to the sample image;

[0024] The second defect probability map corresponding to the sample image is obtained based on the sample image and its corresponding reconstructed image.

[0025] In one embodiment, the step of performing consistency fusion based on the first defect probability map and the second defect probability map to obtain the defect label confidence score of each pixel unit in the sample image includes:

[0026] The value corresponding to each pixel unit in the first defect probability map is the first defect probability of that pixel unit, which is used to represent the probability that the pixel unit belongs to a defect as output by the first teacher model;

[0027] The value corresponding to each pixel unit in the second defect probability map is the second defect probability of that pixel unit, which is used to represent the probability that the pixel unit belongs to a defect as output by the second teacher model;

[0028] Based on the first defect probability and the second defect probability of each pixel unit, the teacher disagreement uncertainty corresponding to that pixel unit is obtained;

[0029] Teacher consensus fusion is performed on the first defect probability, the second defect probability, and the teacher disagreement uncertainty of each pixel unit to obtain the defect label confidence level corresponding to that pixel unit.

[0030] In one embodiment, training a preset student model based on the second sample training set to obtain a student defect detection model includes:

[0031] In the second sample training set, when the defect determination label corresponding to a pixel unit in each sample image indicates that the pixel unit belongs to a defect, the pixel unit also corresponds to a defect type label, which is used to indicate the defect type to which the pixel unit belongs;

[0032] Each sample image in the second sample training set is sequentially input into the preset student model to obtain the prediction result for each sample image. In the prediction result corresponding to each sample image, each pixel unit in the sample image corresponds to a defect determination probability and the type probability of the pixel unit belonging to different defect types.

[0033] The student's total loss function is calculated based on each sample image and its prediction result; the student's total loss function includes defect segmentation loss, defect classification loss, sample consistency constraint loss, anomaly scoring and ranking loss, and background preservation constraint loss.

[0034] Among them, the defect segmentation loss is calculated based on the defect determination probability, defect determination label and defect label confidence of each pixel unit in each sample image;

[0035] The defect classification loss is calculated based on the type probability of each pixel unit in each sample image belonging to different defect types, the defect type label, and the defect label confidence.

[0036] Obtain the enhanced image of each sample image after image enhancement transformation, and use the prediction result obtained by the student model for the enhanced image as the enhancement prediction result corresponding to the sample image; calculate the sample consistency constraint loss based on the prediction result and the enhancement prediction result corresponding to each sample image.

[0037] Obtain the normal sample image corresponding to each counterfactual defect sample image, and calculate the anomaly score ranking loss based on the prediction result corresponding to each counterfactual defect sample image and the prediction result of the normal sample image corresponding to that counterfactual defect sample image.

[0038] The background preservation constraint loss is calculated based on the defect determination probability of the pixel unit belonging to the background region in the prediction result of each sample image.

[0039] The student model is trained according to the total student loss function, and the trained student model is used as the student defect detection model.

[0040] In one embodiment, calculating the anomaly rating ranking loss based on the prediction result corresponding to each counterfactual defect sample image and the prediction result of the corresponding normal sample image includes:

[0041] For any counterfactual defect sample image: based on the defect determination probability of each pixel unit in the counterfactual defect sample image, obtain the first defect score corresponding to the counterfactual defect sample image; based on the defect determination probability of each pixel unit in the normal sample image corresponding to the counterfactual defect sample image, obtain the second defect score corresponding to the normal sample image.

[0042] The abnormal score ranking loss is obtained based on the first defect score and the second defect score, including: calculating the sum of the second defect score and the preset minimum score interval, calculating the difference between the sum and the first defect score; and taking the maximum value between the difference and 0 as the sample loss value corresponding to the counterfactual defect sample image.

[0043] The anomaly score ranking loss is derived from the sample loss value corresponding to all counterfactual defect sample images.

[0044] According to a second aspect, one embodiment provides a method for detecting micro-defects on industrial surfaces, comprising:

[0045] Acquire the image to be tested;

[0046] The image to be tested is input into the student defect detection model to obtain the prediction result of the image to be tested. The prediction result includes the defect determination probability corresponding to each pixel unit and the type probability of the pixel unit belonging to different defect types. The student defect detection model is trained by the model training method for industrial surface micro-defect detection described above.

[0047] Based on the probability of each pixel unit in the image under test belonging to different defect types, the defect type of each pixel unit in the image under test is obtained;

[0048] The detection uncertainty of each pixel unit in the image to be tested is obtained; based on the detection uncertainty of each pixel unit, the defect region in the image to be tested and the defect type corresponding to the defect region are obtained.

[0049] In one embodiment, obtaining the detection uncertainty of each pixel unit in the image to be tested includes:

[0050] A first uncertainty image corresponding to the image to be tested is obtained based on the image to be tested and the student defect detection model; the value corresponding to each pixel unit in the first uncertainty image is used to characterize the inference stability corresponding to that pixel unit.

[0051] A second uncertainty image corresponding to the image to be tested is obtained. The second uncertainty image is obtained during the training process of the student defect detection model. The value corresponding to each pixel unit in the second uncertainty image is used to characterize the basic error degree of the pixel unit.

[0052] The detection uncertainty of each pixel unit is obtained based on the first uncertainty and the second uncertainty of each pixel unit. The reliability of the determination result of whether the pixel unit is a defect is determined based on the detection uncertainty.

[0053] In one embodiment, it further includes:

[0054] For any defective region in the image to be tested: obtain the defect information corresponding to the defective region, the defect information including multiple of the following: area of ​​the defective region, maximum connected length, average contrast, and location risk coefficient corresponding to the defect location;

[0055] The severity of the defect is calculated based on the defect information corresponding to the defect area in the image to be tested.

[0056] According to a third aspect, one embodiment provides a computer-readable storage medium including a program that can be executed by a processor to implement the above-described model training method and industrial surface micro-defect detection method.

[0057] According to the above embodiments, a model training method for detecting micro-defects on industrial surfaces utilizes a counterfactual diffusion model to generate local defects only in designated areas based on normal samples, while maintaining background texture and shooting conditions. This generates defect training samples with clear mask labels, thus constructing trainable data when there are very few real defect samples or even only normal samples, greatly reducing the amount of manual annotation. Through a defect generation mechanism with physical prior constraints, continuity, curvature, scale, and background consistency constraints are introduced for different defect types such as scratches, cracks, dents, and dirt, making the generated defects more consistent with the real morphology of industrial surfaces. Then, a dual-teacher consistency pseudo-label fusion method is used to double-verify the reliability of the defect labels, combining the defect probabilities output by the two teachers and two... The uncertainty of teacher disagreement among teachers is used to configure the confidence of defect labels, thereby adjusting the contribution weight of each pixel unit in each sample image during the training process of the student defect detection model, thus generating a more reliable second sample training set. The lightweight student model is trained based on this second sample training set. During this process, based on the ranking training mechanism between counterfactual defect samples and normal samples, the student model can not only learn defect features, but also learn the abnormal rating differences between the normal and defective states of the same product, thereby improving the recognition ability of the student defect detection model in small sample scenarios. Based on the sample consistency constraint loss, the student defect detection model can adapt well to the detection fluctuations caused by changes in lighting and texture, thus obtaining a more accurate and reliable micro-defect detection model.

[0058] According to the above embodiments, an industrial surface micro-defect detection method uses a lightweight student defect detection model trained by the above model training method to detect defects in the image under test. This model initially identifies defect regions and their types within the image, meeting the real-time detection requirements of edge equipment on the production line. The method evaluates the inference stability of the image under test through multi-scale and / or multiple inferences, and calculates the detection uncertainty of the image under test by combining historical divergence information of pixel units corresponding to each coordinate during training. Based on the detection uncertainty, the reliability of the student defect detection model's defect detection results is determined, achieving uncertainty suppression. This allows for automatic defect identification or manual verification. When the manual verification result differs from the defect detection model's result, the sample is collected for online or offline model updates, achieving closed-loop optimization of the model. Quantifying the severity of defects in the defect region makes the defect detection results more intuitive. Furthermore, by associating the obtained defect detection results with process parameters, production line batches, equipment numbers, etc., defect traceability and process parameter optimization are supported. Attached Figure Description

[0059] Figure 1 This is a flowchart of a model training method for detecting micro-defects on industrial surfaces.

[0060] Figure 2 This is a flowchart of a method for detecting micro-defects on industrial surfaces. Detailed Implementation

[0061] The present invention will now be described in further detail with reference to specific embodiments and accompanying drawings. Similar elements in different embodiments are referred to by associated similar element reference numerals. In the following embodiments, many details are described to facilitate a better understanding of the invention. However, those skilled in the art will readily recognize that some features may be omitted in different situations, or may be replaced by other elements, materials, or methods. In some cases, certain operations related to the present invention are not shown or described in the specification. This is to avoid obscuring the core parts of the invention with excessive description. For those skilled in the art, detailed description of these related operations is not necessary; they can fully understand the related operations based on the description in the specification and general technical knowledge in the art.

[0062] Furthermore, the features, operations, or characteristics described in the specification can be combined in any suitable manner to form various embodiments. At the same time, the steps or actions in the method description can be rearranged or adjusted in a manner obvious to those skilled in the art. Therefore, the various orders in the specification and drawings are only for the clear description of a particular embodiment and do not imply a necessary order, unless otherwise stated that a particular order must be followed.

[0063] The serial numbers assigned to components in this document, such as "first" and "second," are used only to distinguish the described objects and have no sequential or technical meaning. The terms "connection" and "linkage" used in this invention, unless otherwise specified, include both direct and indirect connections (linkages).

[0064] Industrial surface defect detection is a crucial application of machine vision technology in the manufacturing sector. Traditional detection methods primarily rely on threshold segmentation, edge detection, template matching, frequency domain filtering, and manually designed features to identify defects. These algorithms are highly dependent on lighting conditions, workpiece placement, and the regularity of background textures. When the workpiece surface has brushed lines, reflections, oil stains, embossed marks, inherent natural textures, or minor scratches, it is prone to false positives and false negatives.

[0065] Deep learning technology can autonomously learn defect features based on data samples. However, industrial defect data generally suffers from small sample sizes and long-tail distribution: some high-risk defects occur infrequently, sample collection cycles are lengthy, and defect morphology is affected by multiple factors such as material properties, processing technology, lighting environment, and shooting angle. Manually labeled datasets can hardly exhaust all defect scenarios. If a detection model is trained directly using limited defect samples, it is easy to cause overfitting; while if only normal samples are used to train an anomaly detection model, normal texture fluctuations are easily identified as defects.

[0066] While generative models can be used to augment defect samples, general image generation algorithms are prone to destroying the original background texture of the product, generating false defect samples that violate physical reality and do not match the actual conditions on the production line. Although visual basic models have excellent segmentation performance, when dealing with industrial scenarios with minor defects, they are still prone to identifying inherent textures, reflections, object edges, and shadow interference as defects.

[0067] Therefore, this invention targets industrial surfaces such as metals, glass, plastics, ceramics, PCBs, lithium battery electrodes, and automotive parts. During training, the normal texture distribution of the product surface is first learned through a large number of normal samples (i.e., the first sample training set). Then, a physically constrained counterfactual diffusion model generates local defect regions on the normal samples, obtaining counterfactual defect sample images corresponding to the normal sample images. This generates a paired training dataset (i.e., the second sample training set) containing "normal sample image - counterfactual defect sample image - defect mask". Subsequently, a dual-teacher model is introduced, and through dual-teacher consistency verification and uncertainty suppression, the training dataset is improved for each sample in the second sample training set. The defect labels are quality-verified to obtain defect label confidence levels. This allows for the construction of trainable data even when there are very few real defect samples or only normal samples. This reduces the amount of manual annotation and solves the problems of few samples, large morphological variations, complex background textures, and high annotation costs for minor defects such as scratches, indentations, dents, dirt, ablation, cracks, foreign objects, and uneven coatings. Finally, a lightweight student defect detection model is trained based on the defect label confidence levels. Through counterfactual diffusion sample generation and dual-teacher consistency verification, while ensuring normal backgrounds and locally generated reasonable defects, the reliability of defect labels is verified by multiple teacher models to train a more reliable student defect detection model.

[0068] In actual testing, the student defect detection model is used to detect micro-defects on industrial surfaces. Furthermore, the uncertainty mechanism is used to control false alarms and false negatives, thereby improving the recall rate and false alarm control capability of micro-defect detection under complex texture backgrounds. It is applicable to applications such as online production line inspection, appearance quality inspection, defect traceability, and process parameter optimization. It realizes the generation, identification, segmentation, verification, and closed-loop optimization of micro-defects on industrial surfaces under small sample conditions.

[0069] Please refer to Figure 1 Some embodiments provide a model training method for detecting micro-defects on industrial surfaces, including:

[0070] Step S100: Obtain the first sample training set.

[0071] This embodiment first collects surface images of normal samples (i.e., normal sample images) as the first sample training set. A small number of real defect sample images can also be collected as a supplement. That is, if real defect sample images exist in the first sample training set, the number of real defect sample images must be much smaller than the number of normal sample images. Each image records information such as product type, material type, lighting angle, camera exposure, and workstation number.

[0072] In this embodiment, after obtaining the first sample training set, a preset model is trained based on the obtained first sample training set to learn the texture distribution of normal sample images. For example, the network architecture of the preset model is a decoder-decoder architecture. After the sample images in the first sample training set are input into the model, the model outputs the reconstructed image corresponding to the sample image. The mean absolute error loss function (MAE) is calculated based on the input sample image and the output reconstructed image. The network parameters are iteratively updated according to the loss function. After the loss function converges, the trained texture reconstruction model is obtained.

[0073] Step S110: Train the preset diffusion model based on the first sample training set to obtain the counterfactual diffusion model; the counterfactual diffusion model is used to generate counterfactual defect sample images based on the normal sample images in the first sample training set.

[0074] To alleviate the problem of insufficient defect samples, this embodiment further constructs a counterfactual diffusion model, which generates changes that conform to the physical morphology of defects only in a specified local area while keeping the original product background, texture, structure and shooting conditions unchanged.

[0075] During the training process of the counterfactual diffusion model: For any normal sample image in the first sample training set: the normal sample image and its corresponding preset defect type, preset defect location mask and preset defect intensity parameter are input into the diffusion model to obtain the predicted noise corresponding to the normal sample image; based on the predicted noise corresponding to the normal sample image; wherein, the defect location mask is used to determine the generation range of the defect, and the defect intensity parameter is a hyperparameter set by the user to control the significance of the generated defect (e.g., scratch depth, pit amplitude, etc.), and its value range is [0.5, 2.0]. In this embodiment, the default defect intensity parameter is 1.0, and the specific value can be adjusted according to actual needs.

[0076] Then, by using analytical formulas, the counterfactual defective sample image of the normal sample image is derived. Therefore, for an input normal sample image x, the counterfactual defective sample image x' generated based on the diffusion model can be expressed as:

[0077] x'=F θ (x, m, t, ρ, z)

[0078] Where x' represents a counterfactual defective sample image generated based on a normal sample image x; F θ θ represents the preset diffusion model, m represents the preset defect location mask corresponding to the current normal sample image, t represents the preset defect type identifier, ρ represents the preset defect intensity hyperparameter, and z represents the random noise added to the current normal sample image, i.e., the real noise.

[0079] In this embodiment, the loss function during training includes the main loss function and the constraint loss. The main loss function is used to teach the conditional diffusion model how to correctly denoise. In this embodiment, the main loss function is calculated based on the real noise and predicted noise corresponding to each normal sample image. The main loss function can be expressed as:

[0080]

[0081] Among them, L diff Let ε represent the main loss function. i ε represents the true noise corresponding to the i-th normal sample image; i (θ) represents the predicted noise corresponding to the i-th normal sample image when the network parameter of the diffusion model is θ; N represents the total number of normal sample images.

[0082] The constraint loss includes background consistency constraint loss and defect morphology constraint loss. To ensure that the generated image only changes within a preset defect region—that is, to ensure that the parts of the counterfactual defect sample image other than the preset defect generation range remain consistent with the original normal sample image—this embodiment calculates the background consistency constraint loss based on the defect location mask corresponding to the normal sample image and the counterfactual defect sample image. This background consistency constraint loss can be expressed as:

[0083]

[0084] Among them, L bg The background consistency constraint loss is represented by m, where N represents the total number of normal sample images; i Let represent the defect location mask corresponding to the i-th normal sample image. Within this defect location mask, pixels within the allowed defect generation area have a value of 1; "1" represents a matrix of all 1s, 1-m. iThis corresponds to the background area in the mask at the defect location; x i ' represents the counterfactual defective sample image corresponding to the i-th normal sample image; x i This represents the i-th normal sample image; "" represents the Hadamard product, which is the element-wise product.

[0085] To ensure that the defect area conforms to the physical morphology of scratches, dents, cracks, etc., this embodiment further calculates the defect morphology constraint loss based on the counterfactual defect sample image to guarantee the physical morphology of the defect area in the counterfactual defect sample image. The defect morphology constraint loss in this embodiment includes continuity constraint, curvature constraint, and size constraint. The continuity constraint is used to constrain the continuity of linear defects in the counterfactual defect sample image. For example, the total variational loss can be used to penalize pixel-level breaks and jagged edges in the generated defect area, keeping the linear defects continuous. The curvature constraint is used to constrain the smoothness of the bending of defects in the counterfactual defect sample image. For example, the second derivative or curvature loss can be used to constrain the smoothness of the bending of cracks and scratches to avoid abnormal bending or twisting. The size constraint is used to constrain the defect size range (e.g., connected region area) of defects in the counterfactual defect sample image. For example, the L2 loss is calculated between the connected region area of ​​the actual generated defect area and the target area, or the proportion of the actual generated defect area relative to the entire image is controlled within the target range. The defect morphology constraint loss can then be expressed as:

[0086] L shape =λ l L line +λ c L curv +λ s L scale

[0087] Among them, L shape L represents the defect morphological constraint loss. line L represents a continuity constraint. curv Indicates curvature constraint; L scale Indicates dimensional constraints; λ l , λ c , λ s These represent the weights corresponding to continuity constraints, curvature constraints, and size constraints, respectively. Their values ​​can be determined based on the type of defect to be generated (i.e., on / off). For example, if the preset defect type corresponding to the current normal sample image is a scratch, then λ is set. l >0, λ c >0, λ s =0; if the preset defect type corresponding to the current normal sample image is pit, then set λ. l =0, λ c =0, λ s>0, and so on; alternatively, all three weights corresponding to all types can be set to positive values, but the proportions can be adjusted according to the defect type. For example, when the preset defect type is scratch, λ can be set. l For a larger value, λ is set when the preset defect type is pit. s The value should be relatively large; alternatively, the two weighting methods mentioned above can be used in combination when processing each normal sample image in the first training set.

[0088] The counterfactual total loss function is obtained based on the main loss function, background consistency constraint loss, and defect morphology constraint loss. The pre-defined diffusion model is then trained based on the counterfactual total loss function, and the trained diffusion model is used as the counterfactual diffusion model.

[0089] It should be noted that, subject to three physical constraints—continuity, curvature, and size—the actual defect region generated by the counterfactual diffusion model usually only occupies a portion of the preset defect location mask (i.e., the range within which defect generation is allowed). For example, a fine crack will not fill the entire preset generation area. Therefore, the actual defect region corresponding to the counterfactual defect sample image is not necessarily the same as the preset defect generation area.

[0090] The counterfactual diffusion model in this embodiment cannot learn defect generation solely from normal images; it also requires additional supervision signals. It can either use a small number of real defect samples or defect descriptions as conditions to fine-tune the pre-trained diffusion model, or it can perform end-to-end training of the conditional diffusion model when the real defect data has mask annotations.

[0091] Unlike conventional methods that involve rotating, cropping, or adding noise to existing images, this embodiment uses normal samples as a base and generates local defects only in designated areas while maintaining background texture and shooting conditions. This constructs defect training samples with clear mask labels. During the training of the counterfactual diffusion model, a defect generation mechanism with physical prior constraints is used. For different defect types such as scratches, cracks, dents, and dirt, constraints on continuity, curvature, scale, and background consistency are introduced to make the generated defects more consistent with the real morphology of industrial surfaces. This generates locally controllable, morphologically interpretable, and background-consistent defect samples in normal sample images, enabling the subsequently trained model to learn the differences between "the same product with and without defects".

[0092] Step S120: Obtain the second sample training set based on the first sample training set and all counterfactual defect sample images; obtain the pixel-level defect label confidence of each sample image in the second sample training set through a dual-teacher model.

[0093] Based on the trained counterfactual diffusion model, defect samples are expanded. In this embodiment, each normal sample image in the first sample training set is input into the counterfactual diffusion model to obtain a counterfactual defect sample image corresponding to each normal sample image. At the same time as obtaining the counterfactual defect sample image of the normal sample image, a defect mask corresponding to the counterfactual defect sample image is also obtained, thus forming a triplet of "counterfactual defect sample image - normal sample image - defect mask". The defect mask is used to determine the actual generated defect region in the counterfactual defect sample image. It contains a defect determination label corresponding to each pixel unit in the counterfactual defect sample image. The defect determination label is used to characterize whether the corresponding pixel unit belongs to a defect, thereby determining whether each pixel unit in the counterfactual defect sample image belongs to the background region or the defect region. For example, if the pixel unit belongs to a defect, the corresponding defect determination label can be set to 1, and otherwise, the corresponding defect determination label can be set to 0.

[0094] It should be noted that a pixel unit can correspond to a single pixel in an image, or it can aggregate several pixels into a basic processing unit; in this embodiment, one pixel unit corresponds to one pixel.

[0095] Furthermore, since a normal sample image can be pre-set with different defect location masks, defect intensity parameters, or defect types, a normal sample image may correspond to multiple counterfactual defect sample images, but a counterfactual defect sample image corresponds to only one normal sample image.

[0096] All sample images in the first sample dataset (including a large number of normal sample images and a small number of real sample images) and all counterfactual defect sample images together constitute the second sample training set, which serves as the static training data for the subsequent dual-teacher model and student model. A portion of the data in the second sample training set can be used as the validation set.

[0097] It should be noted that the defect determination label can be understood as which pixels in the normal sample image the counterfactual diffusion model modified. It is a hard label that is either 0 or 1. However, the quality of the defect determination label varies. For example, the generated defects may have some unnatural positions or not conform to the true defect shape. If it is directly used as the training label for the subsequent model, it will introduce noise. Therefore, this embodiment introduces a dual-teacher model to verify the quality of the defect determination label: for regions that both teachers agree are defects, the defect determination label corresponding to the region is given a higher confidence weight (i.e., defect label confidence). For regions where the two teachers have a large disagreement, a lower confidence weight is set. At this time, since the two teachers output defect probabilities (corresponding to a numerical range of 0~1), the probability values ​​output by the two teachers can also be regarded as a kind of "pseudo-label", which belongs to soft labels. Compared with the hard labels output by the counterfactual diffusion model, it contains richer edge transition information to further supplement whether the defect determination label is reliable.

[0098] The dual-teacher model in this embodiment includes a first teacher model and a second teacher model. For example, the first teacher model is a visual semantic teacher, which is responsible for discovering suspected defective regions in the input sample image. It can use a visual basic model or segmentation model (such as SAM, CLIP, DINO, etc.) to identify defects in the sample image. The second teacher model is a normal texture teacher, which is responsible for determining whether the input sample image deviates from the normal texture distribution. It obtains abnormal texture response regions by determining whether the texture in the sample image conforms to the texture distribution of normal samples.

[0099] Specifically, for any sample image in the second training set: the first teacher model performs defect semantic recognition on the sample image to obtain the first defect probability map corresponding to the sample image. The value corresponding to each pixel unit in the obtained first defect probability map is the first defect probability of that pixel unit, which is used to represent the probability that the pixel unit belongs to a defect as output by the first teacher model. The teacher model can directly perform zero-shot inference, that is, use the original segmentation capability of the basic model to identify defects without fine-tuning, which is suitable for scenarios with regular textures and obvious differences between defects and backgrounds. Alternatively, it can be used after fine-tuning the pre-trained model, for example, by fine-tuning the basic model on a small number of labeled samples on the production line, thereby improving the ability to identify specific defect morphologies. The specific usage method can be set according to the production line data.

[0100] The sample image is textured using the second teacher model to obtain the second defect probability map corresponding to the sample image. The value of each pixel unit in the second defect probability map is the second defect probability of that pixel unit, which is used to represent the probability that the pixel unit belongs to a defect as output by the second teacher model.

[0101] Specifically, firstly, a pre-trained texture reconstruction model is obtained. This model is trained on a preset model using a first sample training set in step S100 to learn the normal texture distribution. The sample image is then input into the texture reconstruction model to obtain the reconstructed image corresponding to the sample image. Subsequently, a second defect probability map corresponding to the sample image is obtained based on the sample image and its corresponding reconstructed image. For example, the reconstruction error corresponding to each pixel unit is obtained based on the sample image and its reconstructed image. That is, the absolute value of the difference between the pixel value corresponding to each pixel in the sample image and its corresponding pixel value in the reconstructed image is calculated, and the obtained absolute value is used as the abnormal response value of the pixel, thereby obtaining an abnormal response map. In the obtained abnormal response map, the reconstruction error corresponding to the normal area is smaller, while the reconstruction error corresponding to the abnormal area is larger because it does not conform to the normal texture distribution.

[0102] It should be noted that the second teacher model can be the texture reconstruction model itself, which obtains an anomaly response map based on the reconstructed image it outputs, and then obtains a second defect probability map based on the obtained anomaly response map. Alternatively, a post-processing stage can be added to the texture reconstruction model, such as thresholding, normalization, or a lightweight classification head, to further process the anomaly response map and output a more accurate anomaly response region. When a lightweight classification head is added, the encoder-decoder weights of the reconstruction model can be reused, and its network parameters can be updated synchronously during the training of the subsequent student defect detection model, without having to train the complete reconstruction model from scratch.

[0103] Based on the first defect probability map and the second defect probability map, consensus fusion is performed to obtain the defect label confidence of each pixel unit in the sample image. In this embodiment, the teacher divergence uncertainty corresponding to each pixel unit is first obtained based on the first defect probability and the second defect probability of each pixel unit. Then, teacher consensus fusion is performed based on the first defect probability, the second defect probability, and the teacher divergence uncertainty of each pixel unit to obtain the defect label confidence of the pixel unit. For example, for the u-th pixel unit, its corresponding defect label confidence can be expressed as:

[0104] M(u)=σ(w1l1(u)+w2l2(u)﹣w3U(u))

[0105] Where M(u) represents the defect label confidence level corresponding to the u-th pixel unit; σ represents the Sigmoid function; l1(u) represents the original score (i.e., logit output) of the u-th pixel unit output by the first teacher model, which is the unnormalized score output by its segmentation task head (i.e., classification head). This unnormalized score is substituted into the Sigmoid function to calculate the first defect probability corresponding to the u-th pixel unit; l2(u) represents the original score (i.e., logit output) of the u-th pixel unit output by the second teacher model. If the second teacher model adds a lightweight classification head to the texture reconstruction model to process the anomaly response map, its corresponding original score is the unnormalized score output by the classification head. If the second teacher model directly uses the anomaly response map obtained by the texture reconstruction model, since the anomaly response value is greater than or equal to 0, it needs to be linearly mapped or the baseline value subtracted to convert it into an equivalent logit output that can be positive or negative, and then substituted into the Sigmoid function to calculate the first defect probability corresponding to the u-th pixel unit. Two defect probabilities; U(u) represents the teacher disagreement uncertainty corresponding to the u-th pixel unit, which is defined as the absolute value of the difference between the original score of the u-th pixel unit output by the first teacher model and the original score of the u-th pixel unit output by the second teacher model. However, since the numerical range of the original score is large (usually [-10, +10]), the original score can be appropriately normalized before calculating the difference to ensure that the threshold is interpretable. Therefore, in this embodiment, the teacher disagreement uncertainty corresponding to the u-th pixel unit is... The absolute value of the difference between the first defect probability and the second defect probability corresponding to the u-th pixel; w1, w2, and w3 represent the original scores output by the first teacher model, the original scores output by the second teacher model, and the fusion weights corresponding to the teacher divergence uncertainty, respectively. For example, w1∈[0.4, 0.6], w2∈[0.3, 0.5], w3∈[0.1, 0.3], and the default values ​​are w1=0.45, w2=0.35, and w3=0.2. Their values ​​can be adjusted through the validation set according to the specific production line data.

[0106] When two teacher models both classify the same pixel as a defect, the pixel has a defect label with high confidence. If the first teacher model (visual teacher) considers it a defect but the second teacher model (texture teacher) considers it normal texture, or if the second teacher model (texture teacher) considers it abnormal but the first teacher model (visual teacher) considers it a reflection or shadow (i.e., not a defect), the pixel is placed in the low-confidence set. In subsequent training of the student defect detection model, the contribution weight of the pixel is reduced or it is handed over to manual sampling.

[0107] For example, a preliminary judgment is made based on the classification consistency of the two teacher models: if the defect probabilities corresponding to both teacher models are greater than 0.5, the classification consistency of the two teacher models is considered to be high, and the pixel unit is marked as a high-confidence candidate; otherwise, the classification of the two teacher models is considered to be inconsistent, and the pixel unit is directly assigned to the low-confidence set. Then, for the high-confidence candidates selected above, the degree of divergence is further measured by the teacher divergence uncertainty to further filter them. For example, if the teacher divergence uncertainty is lower than a preset threshold, the two teacher models are considered to have strong consistency and corresponding to high confidence; conversely, if the teacher divergence uncertainty is greater than the preset threshold, such as image-level pseudo-defects (e.g., uneven lighting), even if both teacher models identify a pixel unit as a defect, it needs to be assigned to the low-confidence set or handed over to manual sampling for inspection, so as to reduce false alarms caused by reflections, shadows, and texture changes during the actual detection process.

[0108] In the subsequent training process of the student defect detection model, the corresponding loss function is calculated based on the defect label confidence of each pixel unit. For example, the defect label confidence obtained above can be directly used as the contribution weight of the corresponding pixel unit, or the corresponding contribution weight can be manually set according to the confidence level. For example, for pixel units considered to have high confidence, the corresponding contribution weight is set to 1.0, and for pixel units with low confidence, the corresponding contribution weight is set to a value range of [0.1, 0.3]. The specific value can be adjusted within this value range.

[0109] At this point, each pixel unit of each sample image in the second sample training set corresponds to a defect judgment label and a defect label confidence level. In addition, in the second sample training set, when the defect judgment label of a pixel unit in a sample image indicates that the pixel unit belongs to a defect, the pixel unit also corresponds to a defect type label. The defect type label is used to indicate the defect type to which the pixel unit belongs. The value of the defect judgment label corresponding to the pixel unit in the background region is 0, and its corresponding defect type label is also defaulted to 0. Therefore, it is possible to store only the defect type labels corresponding to the pixel units whose defect judgment labels are not 0. Among them, the defect type label corresponding to the defect region in the counterfactual defect sample image is the defect type preset before generating the defect sample. Since the number of real defect samples is small, their corresponding defect type labels can be manually labeled.

[0110] Step S130: Train a preset student model based on the second sample training set to obtain a student defect detection model, which is used for industrial surface micro-defect detection.

[0111] In this embodiment, after obtaining the second sample training set, a student defect detection model that can be deployed to the edge equipment of the production line is trained. It includes two task branches: a segmentation head and a classification head, which are used for defect segmentation and defect classification, respectively. Each sample image in the second sample training set is sequentially input into the preset student model to obtain the prediction result of each sample image. In the prediction result corresponding to each sample image, each pixel unit in the sample image corresponds to a defect judgment probability and the type probability of the pixel unit belonging to different defect types.

[0112] Then, the total student loss function is calculated based on each sample image and its prediction result. In this embodiment, the total student loss function includes defect segmentation loss, defect classification loss, sample consistency constraint loss, anomaly scoring and ranking loss, and background preservation constraint loss. The contribution weight of each pixel unit is determined based on the confidence of the defect label corresponding to each pixel unit. When calculating the loss, if the pixel unit is the smallest calculation unit, the contribution weight of each pixel unit needs to be combined during the calculation process.

[0113] To ensure the reliability of the defect segmentation results, this embodiment calculates the defect segmentation loss L based on the defect determination probability, defect determination label, and defect label confidence level of each pixel unit in each sample image. seg At this point, the defect determination label indicates whether a pixel unit is a true defect. Based on the difference between the defect determination probability of each pixel unit output by the student model and the defect determination label, combined with the contribution weight corresponding to each pixel unit, the binary classification cross-entropy loss function is calculated to obtain the defect segmentation loss.

[0114] To ensure the reliability of the defect classification results, this embodiment calculates the defect classification loss L based on the type probability of each pixel unit in each sample image belonging to different defect types, the defect type label, and the defect label confidence level. cls At this point, the defect type label represents the true defect type of the pixel unit. Based on the difference between the defect type probability of each pixel unit output by the student model and the defect type label, combined with the contribution weight corresponding to the pixel unit, the cross-entropy loss function is calculated to obtain the defect classification loss.

[0115] Furthermore, to ensure the consistency of the output of each sample image after image enhancement transformation, that is, after the enhancement transformation perturbation, normal areas are still judged as normal areas and defective areas are still judged as defective areas, and normal areas are not judged as defective areas or defective areas are judged as normal areas due to image enhancement transformation, so as to ensure that the same defect can still be stably detected under different illumination enhancement, scale changes or slight viewing angle changes, this embodiment further introduces sample consistency constraints to reduce detection fluctuations caused by illumination and texture changes.

[0116] For example, the enhanced image of each sample image after image enhancement transformation is obtained. The image enhancement transformation used includes, but is not limited to, random brightness / contrast adjustment, Gaussian noise, grayscale histogram equalization, random cropping / scaling, elastic deformation, and Gaussian blurring to simulate changes in production line lighting, vibration, and slight shifts. The prediction result obtained by the student model through this enhanced image is used as the enhancement prediction result corresponding to that sample image. The sample consistency constraint loss is calculated based on the prediction result corresponding to each sample image and the enhancement prediction result. The sample consistency constraint loss can then be expressed as:

[0117] L cons =||S θ (T a (x))﹣T a (S θ (x))||1

[0118] Among them, L cons S represents the sample consistency constraint loss; θ T represents the student model with network parameters θ; a Indicates image enhancement transformation; x represents the current sample image; T a (x) represents the enhanced image of the corresponding sample; S θ (T a (x) indicates enhanced prediction results; T a (S θ (x) represents the image enhancement transformation performed on the prediction result corresponding to the current sample image.

[0119] To ensure that the anomalies between counterfactual defect sample images and their corresponding normal sample images are consistent with reality, this embodiment further introduces anomaly score ranking loss to constrain the anomalies between each group of counterfactual defect samples and their corresponding normal sample images. That is, the anomaly score of the counterfactual defect sample image is greater than the anomaly score of the corresponding normal sample image, and there is a certain gap between their anomaly scores. This allows the student model to learn the physical law that the anomaly score of the defect image should be higher than that of the corresponding normal sample image. By learning defect features, the model can also learn the difference in anomaly scores of the same product in normal and defective states, thereby improving the discrimination ability in small sample scenarios.

[0120] This embodiment obtains the normal sample image corresponding to each counterfactual defect sample image, and calculates the anomaly score ranking loss based on the prediction result corresponding to each counterfactual defect sample image and the prediction result of the normal sample image corresponding to that counterfactual defect sample image.

[0121] For example, for any counterfactual defect sample image: based on the defect determination probability of each pixel unit in the counterfactual defect sample image, the first defect score corresponding to the counterfactual defect sample image is obtained. For example, the probability of whether each pixel unit in the counterfactual defect sample image output by the student model belongs to a defect is obtained, and the maximum value of the defect determination probability corresponding to all pixel units is used as the abnormal score (i.e., the first defect score) of the counterfactual defect sample image, or the average value of the defect determination probability corresponding to all pixel units is calculated, and the obtained average value is used as the abnormal score of the counterfactual defect sample image.

[0122] Based on the defect determination probability of each pixel unit in the normal sample image corresponding to the counterfactual defect sample image, the second defect score corresponding to the normal sample image is obtained, that is, the probability of whether each pixel unit in the normal sample image output by the student model belongs to the defect is obtained; if the first defect score is the maximum value, then the maximum value of the defect determination probability corresponding to all pixel units in the normal sample image is taken as the abnormal score (i.e., the second defect score) of the normal sample image; if the first defect score is the average value, then the average value of the defect determination probability corresponding to all pixel units in the normal sample image is calculated, and the obtained average value is taken as the abnormal score of the normal sample image, so that the calculation method of the abnormal score corresponding to the counterfactual defect sample image and the normal sample image in the same group is consistent.

[0123] Then, based on the first defect score and the second defect score, the anomaly score ranking loss is obtained, including: calculating the sum of the second defect score and the preset minimum score interval, calculating the difference between the sum and the first defect score; taking the maximum value between the difference and 0 as the sample loss value corresponding to the counterfactual defect sample image; and obtaining the anomaly score ranking loss based on the sample loss values ​​corresponding to all counterfactual defect sample images, i.e.:

[0124] L rank =max(0, δ+s(x)﹣s(x'))

[0125] Among them, L rank denoted as the anomaly rating ranking loss; x' represents the counterfactual defect sample image; x represents the normal sample image corresponding to the counterfactual defect sample image; s(x') represents the first defect score corresponding to the counterfactual defect sample image; s(x) represents the defect score of the normal sample image corresponding to the counterfactual defect sample image, i.e., the second defect score; δ represents the minimum rating interval, used to ensure that there is at least δ rating difference between positive and negative samples, avoiding the student model outputting similar scores. Its value range can be set to [0.1, 0.3], and the specific value can be adjusted according to the validation set.

[0126] It should be noted that the anomaly rating ranking loss only constrains the counterfactual defective sample images and their paired normal sample images; unpaired images are not included in the calculation of this ranking loss.

[0127] Furthermore, to ensure that the student model does not identify the background region as a defect, the background preservation constraint loss is calculated based on the defect determination probability of the pixel unit belonging to the background region in the prediction result of each sample image. For example, the pixel units belonging to the background region in each sample image in the second sample training set are obtained, and the sum of the defect determination probabilities of each pixel unit in the background region output by the student model is calculated. The obtained sum is used as the background preservation constraint loss so that the output probability of the student model for the background region approaches zero.

[0128] The student model is trained based on the student's total loss function. The trained student model is then used as the student defect detection model, and industrial surface micro-defect detection is subsequently performed based on this student defect detection model.

[0129] This embodiment uses a counterfactual diffusion model to generate local defects only in designated areas based on normal samples, while keeping the background texture and shooting conditions unchanged. This generates defect training samples with clear mask labels, thus constructing trainable data when there are very few real defect samples or even only normal samples, greatly reducing the amount of manual annotation. During training, a defect generation mechanism with physical prior constraints is used to introduce continuity, curvature, scale, and background consistency constraints for different defect types such as scratches, cracks, dents, and dirt, making the generated defects more consistent with the real morphology of industrial surfaces. Then, a dual-teacher consistency pseudo-label fusion method is used to double-verify the reliability of the defect labels. A visual semantic teacher identifies suspected defect areas in the sample image, and a texture teacher detects whether the sample image conforms to a normal texture distribution. The two teachers are combined... The output defect probability and the teacher disagreement uncertainty between the two teachers are configured to adjust the contribution weight of each pixel unit in each sample image during the training process of the student defect detection model, thereby generating a more reliable second sample training set. The lightweight student model is trained based on this second sample training set. During this process, through the ranking training mechanism between counterfactual defect samples and normal samples (abnormal score ranking loss), the student model can not only learn defect features, but also learn the abnormal score differences between the normal and defective states of the same product, which improves the recognition ability of the student defect detection model in small sample scenarios. Through the sample consistency constraint loss, the student defect detection model can better adapt to the detection fluctuations caused by changes in lighting and texture, thus obtaining a more accurate and reliable micro-defect detection model.

[0130] Please refer to Figure 2Some embodiments disclose a method for detecting micro-defects on industrial surfaces, including:

[0131] Step S200: Obtain the image to be tested; input the image to be tested into the pre-trained student defect detection model to obtain the prediction result of the image to be tested. The prediction result includes the defect judgment probability corresponding to each pixel unit and the type probability of the pixel unit belonging to different defect types.

[0132] In this embodiment, the student defect detection model is obtained through the model training method for industrial surface micro-defect detection described above. The specific training process has been explained in detail in the above embodiments and will not be repeated here.

[0133] Step S210: Based on the type probability of each pixel unit in the image to be tested belonging to different defect types, obtain the defect type of each pixel unit in the image to be tested.

[0134] For example, for any pixel unit: obtain the maximum value among all type probabilities corresponding to the pixel unit, and take the defect type corresponding to the maximum value as the defect type corresponding to the pixel unit.

[0135] Step S220: Obtain the detection uncertainty of each pixel unit in the image to be tested; based on the detection uncertainty of each pixel unit, obtain the defect area in the image to be tested and the defect type corresponding to each defect area.

[0136] To control false alarms, this embodiment uses an uncertainty suppression mechanism to determine the reliability of defect detection results. This embodiment performs multi-scale inference and / or multiple inferences on the image under test, calculating the inference stability based on the defect probability corresponding to each pixel unit at different scales, serving as the first indicator for judging the reliability of defect identification results. By combining historical divergence information of each pixel position during training as prior information, the degree of basic error is calculated, making it the second indicator for judging the reliability of defect identification results. The detection uncertainty of the defect identification results is comprehensively judged using both the first and second indicators to improve the reliability and accuracy of defect detection results.

[0137] In this embodiment, a first uncertainty image corresponding to the image under test is obtained based on the image under test and the student defect detection model. The value corresponding to each pixel unit in the first uncertainty image is used to characterize the inference stability corresponding to that pixel unit. For example, the inference stability can be obtained by performing multi-scale inference on the image under test, that is, by scaling the image under test to different scales, for example, downsampling the image under test, with downsampling scales including the original image, 0.75 times, 0.5 times, etc.; and / or upsampling the image under test, with upsampling scales including 1.25 times, 1.5 times, etc.; then the images at each scale are input into the student defect detection model to obtain the prediction results of the images at the corresponding scales; and then all scales are restored to the original scale of the image under test. When performing upsampling operation from a low-resolution scale to a high-resolution scale, some pixel units in the image corresponding to the high-resolution scale do not have a directly corresponding defect judgment probability. At this time, the defect judgment probability of these pixel units can be obtained by interpolation. At this point, each pixel unit in the original scale image corresponds to multiple defect determination probabilities. Then, the variance is calculated based on all defect determination probabilities corresponding to each pixel unit, thereby fusing the multi-scale defect determination probabilities. The resulting variance is the inference variance, which is used to characterize the inference stability.

[0138] In addition, inference stability can also be obtained by performing multiple inferences on the image under test. That is, without changing the resolution of the image under test, multiple forward inferences are performed on the image under test using student defect detection models with different random configurations. This yields multiple defect judgment probabilities corresponding to each pixel unit in the image under test. Then, the inference variance is calculated based on all defect judgment probabilities corresponding to each pixel unit, thereby fusing the defect judgment probabilities obtained from multiple inferences.

[0139] Inference stability can also be achieved by performing multiple inferences at multiple scales on the image under test. This involves inputting the image under test at different scales into multiple student defect detection models with different random configurations to obtain multiple defect judgment probabilities corresponding to each pixel unit in the image under test, and then calculating the inference variance.

[0140] In addition, to balance the efficiency and accuracy of defect detection, this embodiment further obtains a second uncertainty image corresponding to the image to be tested. The value corresponding to each pixel unit in the second uncertainty image is used to characterize the basic error level of the pixel unit. It is obtained during the training process of the student defect detection model. The detection uncertainty is calculated by combining the inference stability and the teacher divergence uncertainty in order to calibrate the results and suppress false alarms.

[0141] For example, the second uncertainty can be predicted by adding an extra output head to the student defect detection model and learning the prediction of teacher divergence uncertainty during the training process of the student defect detection model. In the inference stage (i.e., in practical application), the teacher divergence uncertainty of the image to be tested can be directly predicted. Alternatively, during the training process of the student defect detection model, the historical divergence information of each pixel unit can be statistically analyzed offline and used as a priori map. For example, the teacher divergence uncertainty corresponding to each pixel unit in each sample image in the second sample training set can be statistically analyzed, and the average value of all teacher divergence uncertainties corresponding to pixel units at the same coordinate can be calculated as the basic error degree of the pixel unit at that position to measure the magnitude of the inherent bias of the pixel unit.

[0142] The detection uncertainty of each pixel unit is obtained based on its first and second uncertainties. Taking the u-th pixel unit as an example, its detection uncertainty can be expressed as:

[0143]

[0144] Where G(u) represents the detection uncertainty of the u-th pixel unit; Var represents the inference variance calculated based on multi-scale inference or multiple inferences under different configurations or multiple inferences at multiple scales; U(u) represents the teacher divergence uncertainty corresponding to the u-th pixel unit, which is the degree of basic error corresponding to that pixel unit; k represents the weighting coefficient, the value range of which can be set to [0.3, 0.7], with a default of 0.5. In actual use, it can be adjusted according to the acceptance level of false alarms and false negatives on the production line. For example, when sensitive to false alarms, it is necessary to prevent the student defect detection model from misidentifying the inherent bias of each pixel unit as a defect, so the weighting coefficient needs to be increased; when sensitive to false negatives, it is necessary to make the student defect detection model more relevant to the image to be tested, so the weighting coefficient needs to be decreased.

[0145] Then, based on the detection uncertainty, the reliability of the determination result of whether the pixel unit is a defect is determined. In this embodiment, the defect determination probability threshold θ is first obtained. prob and uncertainty threshold θ low and θ high θ low and θ high The selection is based on the cumulative distribution of teacher disagreement uncertainty in each pixel unit of the validation set, such as θ. low θ is the 30th percentile. high It is the 70th percentile; when the defect determination probability of a pixel unit is greater than the defect determination probability threshold θ prob When the detection uncertainty of a pixel is less than θ, the pixel is considered a valid defect candidate. lowWhen the uncertainty of a pixel unit is greater than θ, it is considered that the low uncertainty of the defect determination result corresponds to a relatively reliable defect detection result for that pixel unit; when the detection uncertainty of a pixel unit is greater than θ... high When the defect determination result of the pixel unit is considered to have high uncertainty, the defect detection result of the pixel unit is considered unreliable.

[0146] For any pixel in the image under test: if the defect determination probability of the pixel is high and the detection uncertainty is low, that is, the probability that the pixel is a defect is high and the defect determination result is reliable, then the pixel is automatically determined to be a defect; if the defect determination probability of the pixel is high, but the detection uncertainty is also high, that is, the probability that the pixel is a defect is high but the defect determination result is unreliable, then the pixel is marked as a suspected defect and manual review is triggered; if the defect determination probability of the pixel is low and the detection uncertainty is also low, then the pixel is not a defect and the defect determination result is reliable, then the pixel is not a defect; if the defect determination probability of the pixel is low... However, the detection uncertainty is high, meaning that the student defect detection model may not consider it a defect, but it may actually be a defect. For example, there may be a lack of such defect samples in the training data, which may prevent the model from effectively learning its discriminative features. Consequently, it may be difficult to identify the unseen defect pattern during the inference stage. In this case, the sample needs to be added to the learning queue for updating the student defect detection model. Specifically, after the sample images in the learning queue are manually reviewed, the samples that are indeed considered to be defects by the manual review are assigned to the incremental training set for online updating of the student defect detection model. If necessary, the texture reconstruction model can also be fine-tuned. In addition, during incremental updates, catastrophic forgetting can be mitigated by mixing with historical samples.

[0147] Similarly, each pixel unit in the image under test is judged to obtain the pixel unit that belongs to the defect. The defect region is extracted by means of connected component analysis, that is, the defect location is determined.

[0148] Step S230: Quantify the severity of each defect area to obtain the defect severity of each defect area.

[0149] In this embodiment, for any defective region in the image to be tested: the defect information corresponding to the defective region is obtained, including multiple factors such as the area A, maximum connectivity length L, average contrast C, and location risk coefficient R corresponding to the defect location. The location risk coefficient is determined according to the location of the defective region on the product. For example, defects located on load-bearing surfaces and critical sealing surfaces have the highest risk, so their location risk coefficient can be set to 1.0; defects located on edge areas have a relatively high risk, so their location risk coefficient can be set to a range of 0.6 to 0.8; while defects located on non-working surfaces or appearance surfaces have the lowest risk, so their location risk coefficient can be set to a range of 0.3 to 0.5; and so on.

[0150] In practical applications, risk mapping tables can be pre-configured according to product type to establish a mapping relationship between defect location and corresponding risk coefficient. After extracting the defect area in the image to be tested, the corresponding location risk coefficient is determined based on the location information of the defect area, thereby assessing the severity of the defect.

[0151] Then, the defect severity is calculated based on the defect information corresponding to the defect area in the image to be tested. For example, the defect severity of the current defect area can be defined as:

[0152] S def =μ1A+μ2L+μ3C+μ4R

[0153] Among them, S def This indicates the severity of the defect in the current defect area; μ1, μ2, μ3, and μ4 represent the reference weights corresponding to different defect information, which satisfy the normalization condition, that is, the sum of these four reference weights is 1. The specific values ​​can be set according to the product's sensitivity to defect area, length, contrast, and location. The default values ​​for μ1, μ2, μ3, and μ4 are 0.3, 0.2, 0.3, and 0.2, respectively. In addition, when the product is more sensitive to minor defects, μ3 can be increased.

[0154] The defect level corresponding to each defect area is determined based on the severity of the defect, such as four levels: minor, moderate, severe, or fatal defect. The defect detection results are then associated with process parameters, production line batch, and equipment number.

[0155] In some embodiments, the overall defect level of the current product can also be assessed based on the defect severity corresponding to all defect areas. For example, the sum of the defect severity corresponding to all defect areas can be calculated, the defect level of the current product can be determined based on the obtained sum, and the current product can be processed accordingly based on the defect level.

[0156] It should be noted that if the results of manual review are inconsistent with the results of the student defect detection model, the sample will be added to the incremental training set, and the generation parameters of the counterfactual diffusion model and the student defect detection model will be updated offline in batches based on the incremental training set.

[0157] This embodiment uses a pre-trained lightweight student defect detection model to detect defects in the test image, initially identifying defect regions and their types, which can meet the real-time detection needs of production line edge equipment. By performing multi-scale and / or multiple inferences on the test image to evaluate inference stability, and combining historical divergence information of pixel units corresponding to each coordinate during training, the detection uncertainty of the test image is calculated. Based on the detection uncertainty, the reliability of the student defect detection model's defect detection results is determined, achieving uncertainty suppression. This allows for automatic defect identification or manual verification. The results of manual verification are then compared with the detection results of the defect detection model. When discrepancies occur, the sample is collected for online or offline model updates, enabling closed-loop optimization and continuous adaptation to new defect types and process changes. Performance improves over time as the production line runs. A three-tiered system—automatic defect assessment, manual review of suspected defects, and a learning queue—supports collaborative decision-making between automatic assessment and manual review, balancing efficiency and reliability. Furthermore, this embodiment quantifies the severity of defects in the defective region, making defect detection results more intuitive. By linking defect detection results with process parameters, production batches, and equipment numbers, defect traceability and process parameter optimization are supported.

[0158] Some embodiments of the present invention also disclose a storage medium including a program that can be executed by a processor to implement the methods described in any of the embodiments herein.

[0159] In the above embodiments, implementation can be achieved, in whole or in part, by software, hardware, firmware, or any combination thereof. Furthermore, as those skilled in the art will understand, the principles herein can be reflected in a computer program product on a computer-readable storage medium pre-loaded with computer-readable program code. Any tangible, non-transitory computer-readable storage medium may be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CDs, DVDs, Blu-ray discs, etc.), flash memory, and / or the like. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to form a machine, such that instructions executing on the computer or other programmable data processing apparatus can generate means for performing a specified function. These computer program instructions can also be stored in a computer-readable storage medium that can instruct the computer or other programmable data processing apparatus to operate in a particular manner, such that instructions stored in the computer-readable storage medium can form an article of manufacture, including means for implementing the specified function. The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to perform a series of operational steps on the computer or other programmable apparatus to produce a computer-implemented process, such that instructions executing on the computer or other programmable apparatus can provide steps for implementing the specified function.

[0160] This document describes various exemplary embodiments with reference to them. However, those skilled in the art will recognize that changes and modifications can be made to the exemplary embodiments without departing from the scope of this document. For example, various operational steps and components for performing operational steps can be implemented in different ways depending on the specific application or considering any number of cost functions associated with the operation of the system (e.g., one or more steps can be deleted, modified, or combined with other steps).

[0161] While the principles herein have been illustrated in various embodiments, numerous modifications to the structures, arrangements, proportions, elements, materials, and components, particularly suited to specific environments and operational requirements, may be used without departing from the principles and scope of this disclosure. These modifications and other alterations or alterations will be included within the scope of this document. Those skilled in the art will recognize that many changes can be made to the details of the above embodiments without departing from the fundamental principles of the invention.

Claims

1. A model training method for detecting micro-defects on industrial surfaces, characterized in that, include: Obtain the first training set; The preset diffusion model is trained based on the first sample training set to obtain the counterfactual diffusion model; The counterfactual diffusion model is used to generate counterfactual defect sample images based on normal sample images in the first sample training set. Each pixel unit in the counterfactual defect sample image corresponds to a defect judgment label, and the defect judgment label is used to characterize whether the pixel unit belongs to a defect. A second sample training set is obtained based on the first sample training set and all counterfactual defect sample images; The pixel-level defect label confidence score of each sample image in the second sample training set is obtained through a dual-teacher model, which includes a first teacher model and a second teacher model. For any sample image in the second sample training set: the first teacher model is used to perform defect semantic recognition on the sample image to obtain a first defect probability map corresponding to the sample image; the second teacher model is used to perform texture recognition on the sample image to obtain a second defect probability map corresponding to the sample image; consistency fusion is performed based on the first defect probability map and the second defect probability map to obtain the defect label confidence score of each pixel unit in the sample image; at this time, each pixel unit of each sample image in the second sample training set corresponds to a defect judgment label and a defect label confidence score. A preset student model is trained based on the second sample training set to obtain a student defect detection model, which is used for detecting micro-defects on industrial surfaces.

2. The model training method as described in claim 1, characterized in that, The step of training a preset diffusion model based on the first sample training set to obtain a counterfactual diffusion model includes: For any normal sample image in the first sample training set: the normal sample image and its corresponding preset defect type, preset defect location mask and preset defect intensity parameters are input into the diffusion model to obtain the predicted noise corresponding to the normal sample image; the defect location mask is used to determine the generation range of the defect; Based on the predicted noise corresponding to the normal sample image, a counterfactual defect sample image of the normal sample image and a defect determination label for each pixel unit in the counterfactual defect sample image are obtained. The main loss function is calculated based on the actual noise corresponding to each normal sample image and the predicted noise. Calculate the background consistency constraint loss based on the defect location mask corresponding to the normal sample image and the counterfactual defect sample image; The defect morphology constraint loss is calculated based on the counterfactual defect sample image. The defect morphology constraint loss is used to ensure the physical shape of the defect region in the counterfactual defect sample image. The defect morphology constraint loss includes continuity constraint, curvature constraint and size constraint. The continuity constraint is used to constrain the continuity of the defect in the counterfactual defect sample image. The curvature constraint is used to constrain the curvature smoothness of the defect in the counterfactual defect sample image. The size constraint is used to constrain the size range of the defect in the counterfactual defect sample image. The counterfactual total loss function is obtained based on the main loss function, the background consistency constraint loss, and the defect morphology constraint loss. The diffusion model is trained using the counterfactual total loss function, and the trained diffusion model is used as the counterfactual diffusion model.

3. The model training method as described in claim 1, characterized in that, The second teacher model is used to perform texture recognition on the sample image to obtain a second defect probability map corresponding to the sample image, including: A pre-trained texture reconstruction model is obtained, which is obtained by training a preset model on the first sample training set to learn the normal texture distribution. The sample image is input into the texture reconstruction model to obtain the reconstructed image corresponding to the sample image; The second defect probability map corresponding to the sample image is obtained based on the sample image and its corresponding reconstructed image.

4. The model training method as described in claim 1, characterized in that, The step of performing consistency fusion based on the first defect probability map and the second defect probability map to obtain the defect label confidence score of each pixel unit in the sample image includes: The value corresponding to each pixel unit in the first defect probability map is the first defect probability of that pixel unit, which is used to represent the probability that the pixel unit belongs to a defect as output by the first teacher model; The value corresponding to each pixel unit in the second defect probability map is the second defect probability of that pixel unit, which is used to represent the probability that the pixel unit belongs to a defect as output by the second teacher model; Based on the first defect probability and the second defect probability of each pixel unit, the teacher disagreement uncertainty corresponding to that pixel unit is obtained; Teacher consensus fusion is performed on the first defect probability, the second defect probability, and the teacher disagreement uncertainty of each pixel unit to obtain the defect label confidence level corresponding to that pixel unit.

5. The model training method as described in claim 1, characterized in that, The step of training a preset student model based on the second sample training set to obtain a student defect detection model includes: In the second sample training set, when the defect determination label corresponding to a pixel unit in each sample image indicates that the pixel unit belongs to a defect, the pixel unit also corresponds to a defect type label, which is used to indicate the defect type to which the pixel unit belongs; Each sample image in the second sample training set is sequentially input into the preset student model to obtain the prediction result for each sample image. In the prediction result corresponding to each sample image, each pixel unit in the sample image corresponds to a defect determination probability and the type probability of the pixel unit belonging to different defect types. The student's total loss function is calculated based on each sample image and its prediction result; the student's total loss function includes defect segmentation loss, defect classification loss, sample consistency constraint loss, anomaly scoring and ranking loss, and background preservation constraint loss. Among them, the defect segmentation loss is calculated based on the defect determination probability, defect determination label and defect label confidence of each pixel unit in each sample image; The defect classification loss is calculated based on the type probability of each pixel unit in each sample image belonging to different defect types, the defect type label, and the defect label confidence. Obtain the enhanced image of each sample image after image enhancement transformation, and use the prediction result obtained by the student model for the enhanced image as the enhancement prediction result corresponding to the sample image; calculate the sample consistency constraint loss based on the prediction result and the enhancement prediction result corresponding to each sample image. Obtain the normal sample image corresponding to each counterfactual defect sample image, and calculate the anomaly score ranking loss based on the prediction result corresponding to each counterfactual defect sample image and the prediction result of the normal sample image corresponding to that counterfactual defect sample image. The background preservation constraint loss is calculated based on the defect determination probability of the pixel unit belonging to the background region in the prediction result of each sample image. The student model is trained according to the total student loss function, and the trained student model is used as the student defect detection model.

6. The model training method as described in claim 5, characterized in that, The step of calculating the anomaly rating ranking loss based on the prediction result corresponding to each counterfactual defective sample image and the prediction result corresponding to the normal sample image of that counterfactual defective sample image includes: For any counterfactual defect sample image: based on the defect determination probability of each pixel unit in the counterfactual defect sample image, obtain the first defect score corresponding to the counterfactual defect sample image; based on the defect determination probability of each pixel unit in the normal sample image corresponding to the counterfactual defect sample image, obtain the second defect score corresponding to the normal sample image. The abnormal score ranking loss is obtained based on the first defect score and the second defect score, including: calculating the sum of the second defect score and the preset minimum score interval, calculating the difference between the sum and the first defect score; and taking the maximum value between the difference and 0 as the sample loss value corresponding to the counterfactual defect sample image. The anomaly score ranking loss is derived from the sample loss value corresponding to all counterfactual defect sample images.

7. A method for detecting micro-defects on industrial surfaces, characterized in that, include: Acquire the image to be tested; The image to be tested is input into the student defect detection model to obtain the prediction result of the image to be tested. The prediction result includes the defect determination probability corresponding to each pixel unit and the type probability of the pixel unit belonging to different defect types. The student defect detection model is obtained by the model training method as described in any one of claims 1 to 6. Based on the probability of each pixel unit in the image under test belonging to different defect types, the defect type of each pixel unit in the image under test is obtained; The detection uncertainty of each pixel unit in the image to be tested is obtained; based on the detection uncertainty of each pixel unit, the defect region in the image to be tested and the defect type corresponding to the defect region are obtained.

8. The method for detecting micro-defects on industrial surfaces as described in claim 7, characterized in that, The step of obtaining the detection uncertainty of each pixel unit in the image to be tested includes: A first uncertainty image corresponding to the image to be tested is obtained based on the image to be tested and the student defect detection model; the value corresponding to each pixel unit in the first uncertainty image is used to characterize the inference stability corresponding to that pixel unit. A second uncertainty image corresponding to the image to be tested is obtained. The second uncertainty image is obtained during the training process of the student defect detection model. The value corresponding to each pixel unit in the second uncertainty image is used to characterize the basic error degree of the pixel unit. The detection uncertainty of each pixel unit is obtained based on the first uncertainty and the second uncertainty of each pixel unit. The reliability of the determination result of whether the pixel unit is a defect is determined based on the detection uncertainty.

9. The method for detecting micro-defects on industrial surfaces as described in claim 7, characterized in that, Also includes: For any defective region in the image to be tested: obtain the defect information corresponding to the defective region, the defect information including multiple of the following: area of ​​the defective region, maximum connected length, average contrast, and location risk coefficient corresponding to the defect location; The severity of the defect is calculated based on the defect information corresponding to the defect area in the image to be tested.

10. A computer-readable storage medium, characterized in that, Includes a program that can be executed by a processor to implement the method as described in any one of claims 1 to 9.

Images