Defect detection method and device based on generative perturbation, equipment, storage medium and product

Abstract

This application discloses a defect detection method, apparatus, device, storage medium, and product based on generative perturbation, relating to the field of industrial defect detection technology. The defect detection method based on generative perturbation includes: acquiring an image of a workpiece to be tested; detecting the workpiece image according to a defect detection model; locating defect regions and outputting preliminary inspection results; extracting physical attribute vectors of the defect regions; continuously perturbing the physical attributes of the defect regions using the physical attribute vectors as generation conditions to generate a variant image sequence with continuously varying defect severity; inputting the variant image sequence into the defect detection model for secondary detection to determine the boundary attribute state that causes the detection result of the defect detection model to undergo category reversal; calibrating the decision boundary of the defect detection model based on the boundary attribute state; and outputting the final detection result of the defect region based on the calibrated defect detection model. This application solves the problem of insufficient interpretability in defect detection judgment.

Description

Technical Field

[0001] This application relates to the field of industrial defect detection technology, and in particular to defect detection methods, apparatus, equipment, storage media and products based on generative perturbation. Background Technology

[0002] In industrial product appearance defect detection scenarios, end-to-end defect detection solutions based on deep learning are commonly used. A typical implementation of this solution involves: collecting a large number of images of good and defective products, manually annotating them, using the annotated data to train a convolutional neural network to extract features and classify the input images, and finally, the trained detection model directly outputs the category determination result of the workpiece under test and the corresponding confidence score. When it is necessary to understand the basis of the detection model's judgment, related technologies typically use heatmaps or class activation maps to highlight the image regions of interest to the model, thereby helping operators understand the model's focus.

[0003] Specifically, the detection model only outputs a static confidence score. Operators cannot glean from this score the extent to which defects need to be mitigated for the model to classify it as a good product, nor can they understand the tolerance boundaries of the model for defects of different severity levels. When the judgment given by the detection model differs from the operator's subjective experience, there is a lack of a communication tool that can intuitively demonstrate the internal judgment criteria of the detection model, making it difficult to effectively reconcile the inconsistency between human and machine standards.

[0004] To address the issue of insufficient interpretability, improved techniques have emerged that utilize generative adversarial networks (GANs) for data augmentation of defective images. This approach learns the distribution characteristics of defective images through adversarial training and uses random noise vectors to generate new defect samples to expand the training dataset, thereby improving the generalization ability of the detection model to some extent. Some improved techniques also attempt to combine image inpainting to fill in defective regions and generate reference images after defect removal. However, the GANs used in these improved techniques lack precise control over the physical properties of defects during the generation process. They typically only generate images in two extreme states: defective or defect-free, failing to achieve continuous and gradual changes in the physical properties of defects. Consequently, they also cannot effectively support the visualization and quantification of the decision boundaries of the detection model. Summary of the Invention

[0005] The main objective of this application is to provide a defect detection method, apparatus, equipment, storage medium, and product based on generative perturbation, aiming to solve the technical problem of insufficient interpretability of the defect detection model judgment process in related technologies.

[0006] To achieve the above objectives, this application proposes a defect detection method based on generative perturbation, the method comprising: Acquire an image of the workpiece to be tested, perform detection on the image of the workpiece to be tested according to a pre-trained defect detection model, locate the defect area, and output the initial inspection result; Extract the physical attribute vector of the defect region, and continuously perturb the physical attributes of the defect region using the physical attribute vector as the generation condition to generate a variant image sequence with continuously varying defect degree. The variant image sequence is input into the defect detection model for secondary detection to determine the boundary attribute state that causes the detection result of the defect detection model to be flipped. The decision boundary of the defect detection model is calibrated based on the boundary attribute state, and the final detection result of the defect region is output based on the calibrated defect detection model.

[0007] In one embodiment, the step of extracting the physical attribute vector of the defect region includes: Based on the bounding box information of the defect region obtained by the defect detection model, a local defect image is extracted from the image of the workpiece to be tested; The local defect image is input into a physical attribute extraction network, which outputs physical attribute values ​​in multiple dimensions, including at least length, width, and contrast parameters. The physical attribute values ​​of the multiple dimensions are combined into the physical attribute vector; The physical attribute vector is input into the attribute encoder, which maps the physical attribute vector to the latent feature space and generates a set of mutually orthogonal control components, such that adjusting any one of the control components does not affect the other control components.

[0008] In one embodiment, the step of continuously perturbing the physical properties of the defect region using the physical property vector as the generation condition to generate a variant image sequence with continuously varying defect severity includes: Determine the target attribute parameter to be perturbed from the physical attribute vector; Starting with the current value of the target attribute parameter, multiple decreasing intermediate target values ​​are generated sequentially according to a preset step size, and the starting value and all the intermediate target values ​​constitute a target attribute value sequence. For each target attribute value in the target attribute value sequence, conditional input data is constructed, which includes the current target attribute value, the binary mask image of the defect region, and the background structure guiding feature map. The conditional input data are sequentially input into the conditional diffusion generation model, which generates variant images corresponding to the target attribute values ​​at the latent space feature level. The pixel values ​​outside the defect region in each variant image are consistent with the corresponding pixel values ​​in the image of the workpiece to be tested. The variant images are arranged in descending order of the target attribute value sequence to obtain the variant image sequence.

[0009] In one embodiment, the step of inputting the variant image sequence into the defect detection model for secondary detection and determining the boundary attribute state that causes the detection result of the defect detection model to undergo category flipping includes: Each variant image in the variant image sequence is sequentially input into the defect detection model for detection, and the defect category confidence and category determination result corresponding to each variant image are obtained; Compare the category determination results of adjacent variant images. When the category determination result of the earlier variant image is a defective category and the category determination result of the later variant image is a good product category, the interval defined by the target attribute value corresponding to the earlier variant image and the target attribute value corresponding to the later variant image is determined as the decision flip interval. Using the decision reversal interval as the initial search interval, the midpoint value of the initial search interval is taken as the detection attribute value; The conditional diffusion generation model is invoked to generate a probe variant image corresponding to the probe attribute value, and the probe variant image is input into the defect detection model for detection to obtain a detection judgment result; If the detection result is a defect category, the lower limit of the initial search interval is updated to the detection attribute value; if the detection result is a good product category, the upper limit of the initial search interval is updated to the detection attribute value. Repeat the operations of taking the midpoint value, generating the probe variant image, and updating the upper and lower limits of the updated search interval until the length of the updated search interval is less than the preset accuracy threshold. The midpoint value of the final search interval is determined as the boundary attribute state.

[0010] In one embodiment, the step of calibrating the decision boundary of the defect detection model based on the boundary attribute state, and outputting the final detection result of the defect region based on the calibrated defect detection model includes: Calculate the difference between the original value of the target attribute parameter and the state value of the boundary attribute, and use the ratio of the difference to the original value as the defect severity margin; Based on the target attribute value and defect category confidence level corresponding to each variant image in the variant image sequence, an attribute decay curve is generated, and the position of the boundary attribute state is marked in the attribute decay curve; The defect type label, the defect category confidence score, and the defect severity margin output by the defect detection model are combined and output as the final detection result of the defect region.

[0011] In one embodiment, the step of calibrating the decision boundary of the defect detection model based on the boundary attribute state further includes: A sliding adjustment control associated with the target attribute parameter is provided. In response to the user's drag operation on the sliding adjustment control, the current attenuation ratio is obtained and converted into an instantaneous target attribute value. The conditional diffusion generation model is invoked to generate an instantaneous variant image corresponding to the instantaneous target attribute value, and the instantaneous variant image and the real-time detection results of the defect detection model on the instantaneous variant image are displayed. While displaying the instantaneous variant image and the real-time detection results, the system receives user-submitted category judgment opinions through the feedback operation area. Collect multiple category judgment opinions and corresponding instant target attribute values ​​received by the feedback operation area to form a feedback sample set; When the number of records in the feedback sample set reaches a preset threshold, the distribution of opinions indicating good products and opinions indicating defects in the feedback sample set is statistically analyzed. Adjust the decision confidence threshold in the defect detection model, which is used to distinguish between defect categories and good product categories, based on the distribution, so that the decision boundary of the defect detection model is calibrated with the user's judgment.

[0012] Furthermore, to achieve the above objectives, this application also proposes a defect detection device based on generative perturbation, the defect detection device based on generative perturbation comprising: The acquisition module is used to acquire an image of the workpiece to be tested, detect the image of the workpiece to be tested according to a pre-trained defect detection model, locate the defect area, and output the initial inspection result. The generation module is used to extract the physical attribute vector of the defect region, and use the physical attribute vector as the generation condition to continuously perturb the physical attributes of the defect region to generate a variant image sequence with continuously varying defect degree. The determination module is used to input the variant image sequence into the defect detection model for secondary detection and determine the boundary attribute state that causes the detection result of the defect detection model to be flipped. The detection module is used to perform decision boundary calibration on the defect detection model based on the boundary attribute state, and output the final detection result of the defect area based on the calibrated defect detection model.

[0013] Furthermore, to achieve the above objectives, this application also proposes a defect detection device based on generative perturbation, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the defect detection method based on generative perturbation as described above.

[0014] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the defect detection method based on generative perturbation as described above.

[0015] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the defect detection method based on generative perturbation as described above.

[0016] One or more technical solutions proposed in this application have at least the following technical effects: Compared with related technologies that only output static confidence scores without knowing the tolerance boundary of the detection model, this application acquires an image of the workpiece to be tested, detects the image of the workpiece to be tested according to a pre-trained defect detection model, locates the defect region and outputs the initial detection result, extracts the physical attribute vector of the defect region, continuously perturbs the physical attributes of the defect region with the physical attribute vector as the generation condition, generates a variant image sequence with continuously varying defect degree, inputs the variant image sequence into the defect detection model for secondary detection, determines the boundary attribute state that causes the detection result of the defect detection model to be category flipped, calibrates the decision boundary of the defect detection model according to the boundary attribute state, and outputs the final detection result of the defect region according to the calibrated defect detection model. Understandably, this application employs a technical approach combining generative perturbation and decision boundary probing. After the defect detection model completes initial inspection and locates the defect region, it generates a series of variant images with progressively decreasing defect severity by subjecting the physical properties of the defect region to controlled continuous perturbation. This simulates the complete transition process of the defect from a severe state to a mild state and even disappearance. Therefore, based on the critical position where the detection results in the variant image sequence undergo category reversal, the boundary attribute state of the detection model can be accurately determined. Subsequently, based on this boundary attribute state, the detection model is calibrated to achieve decision boundary and obtain the final detection result containing a margin for defect severity. This ultimately completes the complete detection and judgment of the image of the workpiece under test. This solves the technical problems of insufficient interpretability of the defect detection model's judgment process and the inability to visualize and quantify the decision boundary of the detection model in related technologies. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0018] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic flowchart of an embodiment of the defect detection method based on generative perturbation provided in this application; Figure 2 This is a flowchart illustrating Embodiment 2 of the defect detection method based on generative perturbation in this application; Figure 3 This is a flowchart illustrating Embodiment 3 of the defect detection method based on generative perturbation in this application; Figure 4This is a schematic diagram of the module structure of the defect detection device based on generative perturbation according to an embodiment of this application; Figure 5 This is a schematic diagram of the device structure of the hardware operating environment involved in the defect detection method based on generative perturbation in the embodiments of this application.

[0020] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0021] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0022] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0023] The main solution in this application embodiment is: Acquire an image of the workpiece to be tested, perform detection on the image of the workpiece to be tested according to a pre-trained defect detection model, locate the defect area, and output the initial inspection result; Extract the physical attribute vector of the defect region, and continuously perturb the physical attributes of the defect region using the physical attribute vector as the generation condition to generate a variant image sequence with continuously varying defect degree. The variant image sequence is input into the defect detection model for secondary detection to determine the boundary attribute state that causes the detection result of the defect detection model to be flipped. The decision boundary of the defect detection model is calibrated based on the boundary attribute state, and the final detection result of the defect region is output based on the calibrated defect detection model.

[0024] In this embodiment, the present application uses a defect detection device based on generative perturbation as the execution subject. For ease of description, it will be referred to as "device" in detail below.

[0025] Because the defect detection models in related technologies only output static confidence scores, operators cannot know to what extent the defects need to be reduced before the model can classify them as good products, nor can they understand the location of the tolerance boundary of the model for defects of different severity. Furthermore, the improved scheme of using generative adversarial networks for data augmentation can only generate images in two extreme states: defective or defect-free. It cannot achieve continuous and controllable gradual changes in the physical properties of defects, and it is also difficult to provide effective visualization and quantitative support for the decision boundary of the detection model.

[0026] This application provides a solution that upgrades defect detection from traditional static binary classification to a closed-loop detection process that includes decision boundary detection and calibration by introducing a generative perturbation mechanism based on physical attribute vectors. After the detection model completes the initial detection and localization of the defect region, physical attributes such as the length, width, and contrast of the defect region are extracted and a physical attribute vector is constructed. This physical attribute vector is used as a control condition to drive a conditional diffusion generative model to continuously attenuate the attributes of the defect region, generating a series of variant image sequences in which the defect severity gradually decreases from the original severe state until it disappears completely. Subsequently, this variant image sequence is used as a detection medium and re-inputted into the detection model for secondary detection. The boundary attribute state that flips the model's judgment result from the defect category to the good product category is searched and located, and the difference between the original attribute value and the boundary attribute state value is quantified as the defect severity margin. Finally, the judgment confidence threshold of the detection model is calibrated based on this boundary attribute state, so that the model's decision boundary is consistent with the human judgment standard, and a comprehensive detection result including defect type label, confidence value, and defect severity margin is output. Therefore, this application makes the originally implicit, black-box judgment criteria of the detection model explicit into observable and quantifiable boundary attribute states. This not only enables operators to intuitively understand the tolerance boundaries of the model, but also provides a reliable quantitative basis for human-machine standard alignment and dynamic adjustment of process parameters in industrial defect detection scenarios.

[0027] Based on this, embodiments of this application provide a defect detection method based on generative perturbation, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the defect detection method based on generative perturbation in this application.

[0028] In this embodiment, the defect detection method based on generative perturbation includes steps S10 to S40: Step S10: Obtain an image of the workpiece to be tested, detect the image of the workpiece to be tested according to the pre-trained defect detection model, locate the defect area and output the initial inspection result. It should be noted that the image of the workpiece under test refers to a digital image obtained by taking pictures of the workpiece on the production line using image acquisition equipment such as an industrial camera, including but not limited to images of the surface of metal parts, the appearance of electronic components, the surface of glass panels, or the surface of plastic products. The pre-trained defect detection model refers to a deep neural network model pre-trained using defect sample images with labeled categories, including but not limited to object detection models based on convolutional neural network architectures or visual detection models based on transformer architectures. The initial detection result refers to the detection conclusion directly output by the defect detection model after performing one forward inference on the image of the workpiece under test, including but not limited to the bounding box coordinates of the defect region, the defect type label, and the corresponding defect category confidence score.

[0029] Understandably, this step performs preliminary detection on the image of the workpiece under test by calling a pre-trained defect detection model, thereby locating and classifying potential defect areas and providing target areas and reference benchmarks for subsequent physical property extraction and generative perturbation operations.

[0030] Step S20: Extract the physical attribute vector of the defect region, and continuously perturb the physical attributes of the defect region using the physical attribute vector as the generation condition to generate a variant image sequence with continuously varying defect degree. It should be noted that the physical attribute vector refers to a vector representation composed of multiple dimensions of physical attribute values. These physical attribute values ​​are used to quantitatively describe the measurable appearance characteristics of the defect region, including but not limited to length and width parameters representing the geometric scale of the defect, contrast parameters representing the visual salience of the defect, orientation angle parameters representing the spatial orientation of the defect, area parameters representing the coverage area of ​​the defect, and texture roughness parameters representing the surface roughness of the defect. The continuous perturbation refers to a controlled adjustment operation that gradually decreases the selected physical attribute parameters according to a preset decay step size while keeping the background content of the defect region unchanged. Each adjustment generates a variant image corresponding to a different attribute intensity. The variant image sequence refers to an image set composed of multiple variant images arranged in descending order of attribute decay. The variant image sequence fully presents the entire process of the defect region gradually reducing from its original severe state to complete disappearance.

[0031] Understandably, this step, by introducing a generative perturbation mechanism based on physical attribute vectors, extends defect detection from a single static judgment to continuous probing of the defect attribute space, providing a sequence of probe images covering the entire range of defect attribute variations for the decision boundary of the subsequent search and detection model. This implementation uses an attribute encoder to map physical attributes to orthogonal control components, making the perturbation operation on the target attribute parameters independent of other attributes. This avoids introducing unexpected appearance changes due to attribute coupling when generating variant images, thus ensuring that the defect severity changes reflected in the variant image sequence originate solely from the attenuation of the target attribute.

[0032] Step S30: Input the variant image sequence into the defect detection model for secondary detection to determine the boundary attribute state that causes the detection result of the defect detection model to be flipped. It should be noted that the secondary detection refers to re-feeding each variant image in the variant image sequence into the defect detection model for forward inference to obtain the detection conclusion for each variant image under the current attribute strength. The category flip refers to the phenomenon where the defect detection model's category determination of the variant image changes from the defect category to the good product category. The boundary attribute state refers to the critical attribute value corresponding to the target attribute parameter when the category determination result of the defect detection model exactly occurs the category flip. The boundary attribute state characterizes the specific location in the attribute space of the good product determination boundary for the current defect type by the defect detection model.

[0033] Understandably, this step uses variant image sequences as a detection medium to perform a secondary traversal detection on the defect detection model, and utilizes the correspondence between the gradual decay of attributes and the flipping of model judgment results to achieve automated localization of the decision boundary of the detection model. In determining the flipping interval, this implementation uses a pairwise comparison of adjacent variant image category judgment results to first determine the attribute range where the flipping occurs, and then iteratively approximates this range using a binary search strategy. This ensures that the determination process of the boundary attribute state balances search efficiency and localization accuracy, requiring only a limited number of generation and detection iterations to converge the boundary attribute state to within a preset accuracy.

[0034] Step S40: Based on the boundary attribute state, perform decision boundary calibration on the defect detection model, and output the final detection result of the defect region according to the calibrated defect detection model.

[0035] It should be noted that the decision boundary calibration refers to adjusting the decision confidence threshold used to distinguish between defect categories and good product categories in the defect detection model by utilizing the actual decision boundary information revealed by the boundary attribute state, so that the adjusted decision boundary is consistent with the operator's decision experience or process standard requirements. The final detection result refers to the comprehensive detection conclusion formed by combining the defect type label and defect category confidence output by the defect detection model with the defect severity margin. The final detection result includes defect category information, severity information, and margin information from the good product decision boundary.

[0036] Understandably, this step forms a closed loop of detection, probing, and calibration by feeding back the detection results of the boundary attribute state to the judgment criteria adjustment stage of the detection model. This enables the defect detection method based on generative perturbation to not only detect defects but also quantify the degree of distance between the defects and the judgment boundary and calibrate the judgment boundary according to actual needs. This improves the information richness and practicality of the defect detection results and provides data support for the setting of process standards and product quality grading in industrial defect detection scenarios.

[0037] This embodiment provides a defect detection method based on generative perturbation. This application adopts a technical approach combining generative perturbation and decision boundary probing. After the defect detection model completes the initial inspection and locates the defect region, it generates a series of variant images with gradually decreasing defect severity by subjecting the physical properties of the defect region to controlled continuous perturbation. This simulates the complete transition process of the defect from a severe state to a mild state and even disappearance. Therefore, based on the critical position where the detection results in the variant image sequence undergo category flipping, the boundary attribute state of the detection model can be accurately determined. Then, based on this boundary attribute state, the detection model is calibrated to achieve decision boundary and obtain the final detection result including a margin for defect severity. This ultimately completes the complete detection and judgment of the image of the workpiece under test. This solves the technical problems of insufficient interpretability of the defect detection model's judgment process and the inability to visualize and quantify the decision boundary of the detection model in related technologies.

[0038] In one feasible implementation, the step of extracting the physical attribute vector of the defective region includes: Based on the bounding box information of the defect region obtained by the defect detection model, a local defect image is extracted from the image of the workpiece to be tested; The local defect image is input into a physical attribute extraction network, which outputs physical attribute values ​​in multiple dimensions, including at least length, width, and contrast parameters. The physical attribute values ​​of the multiple dimensions are combined into the physical attribute vector; The physical attribute vector is input into the attribute encoder, which maps the physical attribute vector to the latent feature space and generates a set of mutually orthogonal control components, such that adjusting any one of the control components does not affect the other control components.

[0039] It should be noted that the bounding box information refers to the coordinate data output by the defect detection model during preliminary detection, used to identify the location range of the defect region, including but not limited to the coordinates of the center point and the width and height dimensions of the bounding box, or the set of vertex coordinates of the bounding box. The local defect image refers to a sub-image containing only the defect region, obtained by cropping the image of the workpiece under test based on the bounding box information. The physical attribute extraction network refers to a pre-trained neural network module used to regress and output the physical attribute values ​​of the multiple dimensions from the input local defect image, including but not limited to a network structure based on a feature extraction backbone of a convolutional neural network superimposed with a multi-task regression head. The attribute encoder refers to a transformation network that maps the physical attribute vector from the original physical attribute space to the latent feature space. The orthogonal control components refer to a set of mutually independent component representations output by the attribute encoder in the latent feature space, satisfying the constraint that the inner product is zero, such that adjusting the value of any one control component does not cause changes in the values ​​of the other control components.

[0040] Understandably, this implementation extracts a local defect image from the image of the workpiece under test and inputs it into a physical attribute extraction network for processing. This transforms the appearance information of the defect area into multi-dimensional attribute values ​​with clear physical meaning, providing a quantifiable operational object for subsequent generative perturbation. Furthermore, by mapping the physical attribute vector to orthogonal control components through an attribute encoder, the mutual interference between different physical attributes during the generation process is eliminated at the source. This ensures that subsequent continuous perturbation operations on the target attribute parameters can accurately act on the desired single attribute dimension, guaranteeing that the defect severity attenuation process reflected in the variant image sequence is caused only by changes in the target attribute. This provides controllable generation conditions for the accurate detection of boundary attribute states.

[0041] In one feasible implementation, the step of continuously perturbing the physical properties of the defect region using the physical property vector as the generation condition to generate a variant image sequence with continuously varying defect levels includes: Determine the target attribute parameter to be perturbed from the physical attribute vector; Starting with the current value of the target attribute parameter, multiple decreasing intermediate target values ​​are generated sequentially according to a preset step size, and the starting value and all the intermediate target values ​​constitute a target attribute value sequence. For each target attribute value in the target attribute value sequence, conditional input data is constructed, which includes the current target attribute value, the binary mask image of the defect region, and the background structure guiding feature map. The conditional input data are sequentially input into the conditional diffusion generation model, which generates variant images corresponding to the target attribute values ​​at the latent space feature level. The pixel values ​​outside the defect region in each variant image are consistent with the corresponding pixel values ​​in the image of the workpiece to be tested. The variant images are arranged in descending order of the target attribute value sequence to obtain the variant image sequence.

[0042] It should be noted that the target attribute parameter refers to a single physical attribute parameter selected from multiple physical attribute parameters contained in the physical attribute vector, which is the object of control in this round of continuous perturbation operation, including but not limited to length parameters, width parameters, or contrast parameters. The preset step size refers to the magnitude of the decrease in the value of the target attribute parameter between two adjacent perturbation operations. The preset step size is preset according to the detection accuracy requirements, including but not limited to a fixed step size or an adaptive step size that is dynamically adjusted according to the decay process. The target attribute value sequence refers to an ordered set of multiple target attribute values ​​generated starting from the initial value and decreasing sequentially according to the preset step size. The target attribute value sequence covers the complete range of change of the target attribute parameter from its original severity to zero or a preset minimum value.

[0043] It should be noted that the conditional input data refers to the comprehensive control data package fed to the conditional diffusion generation model. This conditional input data encodes the current target attribute value, region positioning information, and background preservation constraints into a control signal format acceptable to the model. The binary mask image refers to a binary image with the same spatial dimensions as the image of the workpiece under test. Pixels within the defect region are assigned a first value to identify the generation region, while pixels outside the defect region are assigned a second value to identify the preservation region. The background structure guidance feature map refers to guidance information extracted from the image of the workpiece under test to constrain the content of non-defect regions to remain unchanged during the generation process. This includes, but is not limited to, edge feature maps, depth feature maps, or multi-scale feature representations from intermediate layers of the encoder. The conditional diffusion generation model refers to a deep generative network based on a diffusion probability model, capable of controlled image generation based on conditional inputs. This includes, but is not limited to, conditional diffusion models based on a stable diffusion architecture and fine-tuned using industrial defect image datasets.

[0044] It is understandable that this implementation decomposes the continuous perturbation operation into four sub-steps: target attribute parameter selection, attenuation sequence construction, conditional data assembly, and frame-by-frame generation of the conditional diffusion generation model, thus constructing a complete and controllable mapping channel from the physical attribute space to the image space. This implementation combines the current target attribute value, binary mask image, and background structure-guided feature map as conditional inputs. This allows the conditional diffusion generation model to generate variant images according to specified attribute intensities when performing controlled redrawing of the defect region at the latent space feature level. Simultaneously, the background structure-guided feature map ensures that the pixel content of non-defect regions remains strictly consistent with the original image of the workpiece under test. Therefore, the differences between variant images in the variant image sequence originate only from the attenuation of the target attribute parameter, providing a clean and interference-free detection medium for subsequent decision boundary detection based on the variant image sequence.

[0045] In one feasible implementation, the step of inputting the variant image sequence into the defect detection model for secondary detection and determining the boundary attribute state that causes the detection result of the defect detection model to undergo category flipping includes: Each variant image in the variant image sequence is sequentially input into the defect detection model for detection, and the defect category confidence and category determination result corresponding to each variant image are obtained; Compare the category determination results of adjacent variant images. When the category determination result of the earlier variant image is a defective category and the category determination result of the later variant image is a good product category, the interval defined by the target attribute value corresponding to the earlier variant image and the target attribute value corresponding to the later variant image is determined as the decision flip interval. Using the decision reversal interval as the initial search interval, the midpoint value of the initial search interval is taken as the detection attribute value; The conditional diffusion generation model is invoked to generate a probe variant image corresponding to the probe attribute value, and the probe variant image is input into the defect detection model for detection to obtain a detection judgment result; If the detection result is a defect category, the lower limit of the initial search interval is updated to the detection attribute value; if the detection result is a good product category, the upper limit of the initial search interval is updated to the detection attribute value. Repeat the operations of taking the midpoint value, generating the probe variant image, and updating the upper and lower limits of the updated search interval until the length of the updated search interval is less than the preset accuracy threshold. The midpoint value of the final search interval is determined as the boundary attribute state.

[0046] It should be noted that the defect category confidence score refers to the probability value output by the defect detection model after performing forward inference on the input variant image, representing that the variant image belongs to the defect category. The category determination result refers to the binary classification conclusion obtained by the defect detection model after comparing the defect category confidence score with a preset determination confidence threshold. When the defect category confidence score is higher than the determination confidence threshold, the category determination result is a defect category; when the defect category confidence score is lower than or equal to the determination confidence threshold, the category determination result is a good product category. The decision flip interval refers to the attribute value interval jointly defined by the target attribute value corresponding to the earlier variant image and the target attribute value corresponding to the later variant image when the category determination result of two adjacent variant images in the variant image sequence first changes from the defect category to the good product category.

[0047] It should be noted that the probe attribute value refers to the intermediate target attribute value generated within the decision flip interval using a binary search strategy for tentative generation and detection in order to accurately locate the boundary attribute state. The probe variant image refers to the verification image generated by calling the conditional diffusion generation model with the probe attribute value as the target attribute value, used to detect the judgment result of the defect detection model under the attribute intensity. The detection judgment result refers to the category judgment conclusion obtained after inputting the probe variant image into the defect detection model. The preset accuracy threshold refers to the upper limit of the search interval length preset to control the search termination condition. The preset accuracy threshold determines the positioning accuracy of the boundary attribute state, and its value is determined according to the accuracy requirements of the actual detection scenario.

[0048] It is understood that this implementation divides the process of determining the boundary attribute state into two stages: preliminary localization and precise search. The preliminary localization stage utilizes pairwise comparisons of the category determination results of adjacent variant images in the variant image sequence to quickly pinpoint the decision flip interval where category flipping occurs, avoiding blind searching within the entire attribute value range. The precise search stage employs a binary search strategy within the decision flip interval. It repeatedly calls the conditional diffusion generation model to generate probe variant images and progressively narrows the search interval based on the probe determination results. This ensures that the boundary attribute state localization process combines high search efficiency with accurate results, requiring only a limited number of generation and detection iterations to converge the boundary attribute state to the range defined by a preset accuracy threshold. This implementation establishes a closed-loop calling relationship between the conditional diffusion generation model and the defect detection model. The generation model serves as a sampling tool for the attribute space, while the detection model serves as a boundary position determination tool. Their collaborative work achieves automated quantitative detection of the decision boundaries within the detection model.

[0049] For example, refer to Figure 2The boundary attribute state determination process provided in this application embodiment includes the following steps in sequence: After inputting the variant image sequence image by image into the defect detection model to obtain the category determination result, the category determination results of adjacent variant images are compared pair by pair. When the earlier variant image is in the defect category and the later variant image is in the good product category, the target attribute value interval corresponding to the two adjacent variant images is determined as the decision flip interval. Using this interval as the initial search interval, the midpoint value is taken as the probe attribute value, a probe variant image is generated, and the probe determination result is obtained. The upper or lower limit of the search interval is updated according to the probe determination result, and the iteration is repeated until the interval length is less than a preset accuracy threshold. The midpoint value of the final search interval is determined as the boundary attribute state.

[0050] In one feasible implementation, the step of calibrating the decision boundary of the defect detection model based on the boundary attribute state, and outputting the final detection result of the defect region based on the calibrated defect detection model includes: Calculate the difference between the original value of the target attribute parameter and the state value of the boundary attribute, and use the ratio of the difference to the original value as the defect severity margin; Based on the target attribute value and defect category confidence level corresponding to each variant image in the variant image sequence, an attribute decay curve is generated, and the position of the boundary attribute state is marked in the attribute decay curve; The defect type label, the defect category confidence score, and the defect severity margin output by the defect detection model are combined and output as the final detection result of the defect region.

[0051] It should be noted that the defect severity margin refers to the normalized ratio obtained by dividing the absolute difference between the original value of the target attribute parameter and the state value of the boundary attribute by the original value. The defect severity margin is used to quantitatively measure the buffer space remaining between the current defect and the good product judgment boundary of the defect detection model. The larger the margin value, the farther the current defect is from the judgment boundary and the more severe the defect is; the smaller the margin value, the closer the current defect is to the judgment boundary. The attribute decay curve is a two-dimensional relationship curve formed by connecting or fitting the data points corresponding to each variant image in the variant image sequence sequentially, with the value or decay ratio of the target attribute parameter as the horizontal axis variable and the defect category confidence as the vertical axis variable. The attribute decay curve intuitively shows the monotonically decreasing trend of the defect category confidence of the defect detection model as the target attribute parameter decays. The defect type label refers to the classification label output by the defect detection model in the initial inspection stage to identify the category to which the defect belongs. The defect type label includes, but is not limited to, scratch label, dent label, foreign object label, or crack label.

[0052] Understandably, this implementation converts the detection results of boundary attribute states into two output forms: defect severity margin and attribute decay curve. The former provides operators with a concise and comparable quantitative indicator of defect severity in a normalized numerical manner, while the latter graphically presents the response trajectory of the detection model's confidence level throughout the entire process of the defect's physical attributes decaying from the original state to the boundary state. This allows operators to intuitively understand the detection model's tolerance characteristics and judgment criteria for the current defect type. This implementation combines the defect type label, defect category confidence level, and defect severity margin as the final detection result output. This ensures that the detection conclusion not only includes a qualitative judgment and confidence level regarding the presence or absence of a defect but also information on the margin of the defect's distance from the judgment boundary. This provides multi-dimensional decision-making basis for defect classification and handling, process parameter adjustment, and quality traceability in industrial defect detection scenarios.

[0053] In one feasible implementation, the step of calibrating the decision boundary of the defect detection model based on the boundary attribute state further includes: A sliding adjustment control associated with the target attribute parameter is provided. In response to the user's drag operation on the sliding adjustment control, the current attenuation ratio is obtained and converted into an instantaneous target attribute value. The conditional diffusion generation model is invoked to generate an instantaneous variant image corresponding to the instantaneous target attribute value, and the instantaneous variant image and the real-time detection results of the defect detection model on the instantaneous variant image are displayed. While displaying the instantaneous variant image and the real-time detection results, the system receives user-submitted category judgment opinions through the feedback operation area. Collect multiple category judgment opinions and corresponding instant target attribute values ​​received by the feedback operation area to form a feedback sample set; When the number of records in the feedback sample set reaches a preset threshold, the distribution of opinions indicating good products and opinions indicating defects in the feedback sample set is statistically analyzed. Adjust the decision confidence threshold in the defect detection model, which is used to distinguish between defect categories and good product categories, based on the distribution, so that the decision boundary of the defect detection model is calibrated with the user's judgment.

[0054] It should be noted that the sliding adjustment control refers to a graphical operation component deployed on the human-computer interaction interface for receiving continuous adjustment commands from the user. The sliding adjustment control includes, but is not limited to, horizontal or vertical drag sliders, rotary adjustment components, or touch-sensitive numerical input components. The current attenuation ratio refers to the percentage attenuation of the target attribute parameter relative to its original value at the current position of the slider of the sliding adjustment control. The instantaneous target attribute value refers to the specific numerical value of the target attribute parameter obtained after converting the current attenuation ratio. The instantaneous variant image refers to a variant image generated in real-time by calling the conditional diffusion generation model with the instantaneous target attribute value as the control condition, matching the attribute attenuation level currently set by the user. The real-time detection result refers to the category determination conclusion and corresponding defect category confidence level obtained immediately after inputting the instantaneous variant image into the defect detection model.

[0055] It should be noted that the feedback operation area refers to the interactive area set on the human-computer interaction interface for receiving the user's subjective category judgment on the currently displayed real-time variant image. The feedback operation area includes at least a first feedback option corresponding to good product identification and a second feedback option corresponding to defect identification, including but not limited to graphic buttons, checkboxes, or radio buttons. The category judgment opinion refers to the subjective category assignment label submitted by the user through the feedback operation area for the real-time variant image under a specific real-time target attribute value. The feedback sample set refers to a record set consisting of multiple sets of real-time target attribute values ​​and corresponding category judgment opinions. Each set of records is associated with the attribute decay degree and the user's subjective judgment conclusion at that degree. The preset threshold refers to the minimum number of feedback samples required to trigger the automatic adjustment of the judgment confidence threshold. The value of the preset threshold is pre-configured according to the calibration reliability requirements.

[0056] Understandably, this implementation method, by introducing an interactive mechanism combining a sliding adjustment control and a feedback operation area, expands the decision boundary detection results based on generative perturbation from a one-way output to a two-way human-machine aligned closed loop. Operators can freely explore the defect detection model's judgment response at different defect severity levels within the full range of attribute decay by dragging the sliding adjustment control. When a discrepancy is found between the model's judgment and their own experience, they can immediately express their subjective judgment through the feedback operation area. This implementation method collects feedback opinions to form a sample set and adjusts the judgment confidence threshold based on the sample distribution after reaching a preset threshold. This allows the decision boundary of the detection model to undergo statistically automatic offset correction based on the operator's actual judgment experience, rather than relying on a single subjective judgment or manual parameter setting. This gradually achieves adaptive alignment between the detection model's judgment criteria and human judgment criteria in industrial application environments.

[0057] For example, to help understand the implementation process of the defect detection method based on generative perturbation obtained by combining this embodiment with the above embodiment one, please refer to... Figure 3 , Figure 3 A simplified flowchart of a defect detection method based on generative perturbation is provided, specifically: The overall process is divided into five core stages, and each stage will be explained in detail below in conjunction with the system architecture.

[0058] I. Decoupling Defect Detection and Physical Attribute Extraction: The image of the workpiece to be tested captured by the industrial camera is denoted as I_origin. First, a pre-trained defect detection model performs preliminary detection on I_origin, and a target detection network is used to locate the defect region R_box, outputting defect type labels, such as scratches, dents, foreign objects, etc.

[0059] After obtaining the defect region location information, local defect image patches are cropped from I_origin and fed into the physical attribute extraction network for regression calculation. The physical attribute extraction network outputs a multidimensional physical attribute vector V_attr: V_attr = [L(length), W(width), D(depth or contrast), θ(direction angle), S(area), T(texture roughness)] Where L represents the actual length of the longest span of the defect along the extension direction after conversion by camera calibration parameters, W represents the average width of the defect in the vertical direction, D represents the relative difference ratio between the average pixel value of the defect region and the average pixel value of the background region, θ represents the angle between the main extension direction of the defect and the reference coordinate axis, S represents the area of ​​connected pixels covered by the defect region mask, and T represents the texture roughness calculated by the contrast feature of the local gray-level co-occurrence matrix or the local binary mode feature.

[0060] After extracting V_attr, it is input into the attribute encoder for decoupling mapping, and the output is the independent control vector C_attr in the latent space: C_attr = Encoder_attr(V_attr) = [c_L, c_W, c_D, c_θ, c_S, c_T] The control components are orthogonal to each other and satisfy the constraint that the inner product of each pair is zero, ensuring that the regulation of a single attribute will not affect other attributes.

[0061] Alternative methods can be used to extract physical attribute vectors. When the workpiece geometry is regular and imaging conditions are controlled, traditional image processing operators can be used to perform geometric measurements on the defect area mask to extract physical attribute values.

[0062] II. Generative Perturbation Space Construction: The conditional generation architecture adopts a stable diffusion-based conditional generation framework and improves upon it with multi-condition joint control. Traditional image inpainting methods only use the mask as a condition, i.e., I_output = G(I_input, Mask); traditional generative adversarial networks only use random noise as a condition, i.e., I_output = G(z). The conditional generation method of this application is as follows: I_output = G(I_input, Mask, V_attr', C_control) Where I_input represents the original defect image, Mask represents the defect region mask, V_attr' represents the target attribute vector, and C_control represents the structural control signal from the control network, used to ensure background consistency.

[0063] The specific implementation steps for continuous perturbation of attributes are as follows. First, set the target attribute of the probe, such as contrast, and define the attenuation sequence: V_attr^(i) = V_attr - i·ΔV, i = 0, 1, 2, ..., N Where ΔV is the attribute decay step size, which can be dynamically adjusted according to accuracy requirements. Taking a scratch with a contrast C_0 = 0.8 as an example, the decay sequence is generated as follows: C_0 = 0.8 (original), C_1 = 0.72 (decay 10%), C_2 = 0.64 (decay 20%), and so on, until C_n = 0 (complete elimination), where a certain C_k is the critical point to be detected.

[0064] Then, using a control network and guided by the attribute sequence, variant image sequences are generated: I_gen^(i) = ControlNet(VAE_Encode(I_origin), Mask, V_attr^(i),Prompt) Here, VAE_Encode(I_origin) represents the latent space features obtained by encoding the original image using a variational autoencoder, and Prompt is a text prompt, such as a slight scratch on a metal surface, used to further guide the generation quality.

[0065] An alternative implementation of generative perturbation is to replace the deep generative model with traditional image processing algorithms when computational resources are limited. For example, contrast attenuation can be achieved by adjusting the parameters of the gamma correction curve, and size reduction can be achieved by gradually reducing the coverage area of ​​the defect mask through multiple rounds of morphological erosion operations and filling with the neighborhood mean.

[0066] III. Automatic Decision Boundary Detection: After generating the variant image sequence, the process moves to the decision boundary closed-loop detection stage. Let the original image be I_origin, the attribute vector be V_attr, the detection model be M, the accuracy threshold be ε, and the detection targets be the output critical attribute vector V_critical and the severity margin.

[0067] The detection process is as follows: The first step is to initialize V_low = 0, V_high = V_attr, and obtain the original detection result result_origin = M(I_origin).

[0068] The second step is to return that no further probing is needed if result_origin is a good product, since the original image is already a good product.

[0069] The third step is to perform the following operations in a loop until |V_high - V_low| < ε: take the midpoint value V_mid = (V_low + V_high) / 2; call the generation module to generate I_mid = Generate(I_origin, V_mid); obtain the detection result result_mid = M(I_mid); if result_mid is a defect, then set V_high = V_mid, otherwise set V_low = V_mid.

[0070] The fourth step is to determine the critical value V_critical = V_mid.

[0071] Step 5: Calculate the normalized defect severity margin: Margin = ||V_attr - V_critical|| / ||V_attr||.

[0072] Step 6: Return V_critical and Margin as the detection results.

[0073] Multi-attribute joint detection employs a gradient-guided joint optimization approach to solve the following optimization problem: V_critical* = arg min_V' ||V' - V_attr||, satisfying the constraint M(Generate(I,V')) ≥ τ_OK The goal is to find the boundary attribute vectors that minimize the distance between the attribute vectors of the generated image and the original attribute vectors, while simultaneously ensuring that the generated image is classified as a good product by the detection model. This optimization problem can be solved using the Lagrange multiplier method or projective gradient descent.

[0074] IV. Visualization of Decision Boundaries and Model Calibration: After obtaining the critical attribute vector and the defect severity margin, the process moves on to the decision boundary visualization and model calibration stage.

[0075] Attribute decay curve generation: For each detection process, record the attribute value and detection confidence data pair, and fit and plot the decay curve. Plot the attribute decay ratio on the horizontal axis, with values ​​ranging from zero to 100%, and plot the defect confidence output by the detection model on the vertical axis. Mark the critical point, decision threshold line, and confidence interval on the curve.

[0076] Counterfactual image sequence comparison display: The key node images are displayed side by side, forming a complete evolution process from the original image through the critical point to complete elimination: Original image defect state confidence 0.92, attenuation 25% defect state confidence 0.71, critical point boundary state confidence 0.50, attenuation 50% good state confidence 0.28, complete elimination good state confidence 0.05.

[0077] Multidimensional decision boundary surface: When considering multiple attributes simultaneously, the decay ratio of attribute 1 is used as the horizontal axis, the decay ratio of attribute 2 is used as the vertical axis, and the detection confidence is used as the vertical axis or color mapping dimension to fit and generate a three-dimensional decision boundary surface f(x,y) = τ, where τ is the decision threshold.

[0078] The interactive human-computer alignment interface provides model calibration functionality. The interface includes: The main display area is used to compare the original image with the composite image with the currently selected perturbation level; The attribute slider group has one slider for each controllable attribute, with a decay range of zero to 100%. The detection results panel displays the detection results and confidence level of the current composite image in real time; The decision curve graph dynamically highlights the curve points corresponding to the current slider position. The feedback button group includes three feedback options: This image should be a good product, This image should be a defective product, and The boundary is reasonable.

[0079] The feedback processing mechanism covers two typical calibration scenarios. Scenario A is when the model is too strict: the user slides to a position where they believe the composite image is good, but the model still classifies it as defective. The user clicks on the image and it should be good; the system records the composite image, the good image label, the user's identifier, and a timestamp. Scenario B is when the model is too loose: the user slides to a position where they believe the composite image still has obvious defects, but the model classifies it as good; the user clicks on the image and it should be defective; the system records the composite image, the defect label, the user's identifier, and a timestamp.

[0080] The adaptive calibration strategy includes: after collecting sufficient feedback samples, using soft labels to perform online distillation on the last few layers of the detection model; automatically adjusting the decision threshold τ based on the statistical distribution of user feedback; and storing confirmed boundary cases in the process standard knowledge base as a basis for subsequent training and auditing.

[0081] An alternative implementation of the feedback mechanism is to store user feedback data in a hard sample library instead of directly adjusting the model threshold, for use in the next round of offline fine-tuning of the model.

[0082] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the defect detection method based on generative perturbation in this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0083] This application also provides a defect detection device based on generative perturbation, please refer to... Figure 4 The defect detection device based on generative perturbation includes: The acquisition module 10 is used to acquire an image of the workpiece to be tested, detect the image of the workpiece to be tested according to a pre-trained defect detection model, locate the defect area, and output the initial inspection result. The generation module 20 is used to extract the physical attribute vector of the defect region, and continuously perturb the physical attributes of the defect region using the physical attribute vector as the generation condition to generate a variant image sequence with continuously varying defect degree. The determination module 30 is used to input the variant image sequence into the defect detection model for secondary detection and determine the boundary attribute state that causes the detection result of the defect detection model to be flipped. The detection module 40 is used to perform decision boundary calibration on the defect detection model based on the boundary attribute state, and output the final detection result of the defect area based on the calibrated defect detection model.

[0084] And / or, the defect detection device based on generative perturbation includes: The first inference module is used to infer the image to be tested using the target detection network and output the bounding box information of the defect region and the defect type label. The first cropping module is used to crop a local defect image from the image to be tested based on the bounding box information; The first input module is used to input the local defect image into the physical attribute extraction network and output physical attribute values ​​in multiple dimensions, wherein the physical attribute values ​​in multiple dimensions include at least length parameters, width parameters and contrast parameters. The first combination module is used to combine the physical attribute values ​​of the multiple dimensions into the physical attribute vector; The first mapping module is used to input the physical attribute vector into the attribute encoder, which maps the physical attribute vector to the latent feature space and generates a set of mutually orthogonal control components, such that adjusting any one of the control components does not affect the other control components.

[0085] And / or, the defect detection device based on generative perturbation includes: The first determining module is used to determine the target attribute parameter to be perturbed from the physical attribute vector; The first generation module is used to generate multiple decreasing intermediate target values ​​sequentially according to a preset step size, starting from the current value of the target attribute parameter and all the intermediate target values, thereby forming a target attribute value sequence. The first construction module is used to construct conditional input data for each target attribute value in the target attribute value sequence. The conditional input data includes the current target attribute value, the binary mask image of the defect region, and the background structure guiding feature map. The second input module is used to sequentially input the conditional input data into the conditional diffusion generation model, and the conditional diffusion generation model generates variant images corresponding to the target attribute values ​​at the latent space feature level, and the pixel values ​​in each variant image located outside the defect region are consistent with the corresponding pixel values ​​in the image to be tested. The first arrangement module is used to arrange each of the variant images in descending order of the target attribute value sequence to obtain the variant image sequence.

[0086] And / or, the defect detection device based on generative perturbation includes: The third input module is used to input each variant image in the variant image sequence into the defect detection model in sequence, and obtain the defect category confidence and category determination result corresponding to each variant image; The first comparison module is used to compare the category determination results of adjacent variant images. When the category determination result of the earlier variant image is a defective category and the category determination result of the later variant image is a good product category, the interval defined by the target attribute value corresponding to the earlier variant image and the target attribute value corresponding to the later variant image is determined as the decision flip interval. The first value-taking module is used to take the midpoint value of the initial search interval as the detection attribute value, with the decision flip interval as the initial search interval. The second generation module is used to call the conditional diffusion generation model to generate a probe variant image corresponding to the probe attribute value, and input the probe variant image into the defect detection model to obtain the detection judgment result; The first update module is used to update the lower limit of the initial search interval to the detection attribute value if the detection judgment result is a defect category, and to update the upper limit of the initial search interval to the detection attribute value if the detection judgment result is a good product category. The first repeating module is used to repeatedly perform the operations of taking the midpoint value, generating the probe variant image and updating the upper and lower limits of the updated search interval until the length of the updated search interval is less than the preset accuracy threshold. The second determining module is used to determine the midpoint value of the final search interval as the boundary attribute state.

[0087] And / or, the defect detection device based on generative perturbation includes: The first calculation module is used to calculate the difference between the original value of the target attribute parameter and the boundary attribute state value, and to use the ratio of the difference to the original value as the defect severity margin. The second generation module is used to generate an attribute decay curve based on the target attribute value and defect category confidence level corresponding to each variant image in the variant image sequence, and to mark the position of the boundary attribute state in the attribute decay curve. The first merging module is used to merge the defect type label, the defect category confidence score, and the defect severity margin output by the defect detection model, and output them as the final detection result of the defect region.

[0088] And / or, the defect detection device based on generative perturbation includes: The first providing module is used to provide a sliding adjustment control associated with the target attribute parameter, and in response to the user's drag operation on the sliding adjustment control, obtain the current attenuation ratio and convert it into an instantaneous target attribute value; The third generation module is used to call the conditional diffusion generation model to generate an instant variant image corresponding to the instant target attribute value, and to display the instant variant image and the real-time detection results of the defect detection model on the instant variant image; The first receiving module is used to receive the category judgment opinion submitted by the user through the feedback operation area while displaying the instantaneous variant image and the real-time detection result; The first collection module is used to collect multiple category judgment opinions and corresponding instant target attribute values ​​received by the feedback operation area to form a feedback sample set; The first statistical module is used to statistically analyze the distribution of opinions indicating good products and opinions indicating defects in the feedback sample set when the number of records in the feedback sample set reaches a preset threshold. The first adjustment module is used to adjust the decision confidence threshold in the defect detection model that distinguishes between defect categories and good product categories according to the distribution, so that the decision boundary of the defect detection model is calibrated with the user's judgment.

[0089] The defect detection device based on generative perturbation provided in this application, employing the defect detection method based on generative perturbation in the above embodiments, can solve the technical problem of insufficient interpretability in the defect detection model judgment process in related technologies. Compared with the prior art, the beneficial effects of the defect detection device based on generative perturbation provided in this application are the same as those of the defect detection method based on generative perturbation provided in the above embodiments, and other technical features in the defect detection device based on generative perturbation are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0090] This application provides a defect detection device based on generative perturbation. The defect detection device based on generative perturbation includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the defect detection method based on generative perturbation in the first embodiment described above.

[0091] The following is for reference. Figure 5 The diagram illustrates a structural schematic of a defect detection device based on generative perturbation suitable for implementing embodiments of this application. The defect detection device based on generative perturbation in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, tablets, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital televisions and desktop computers. Figure 5 The defect detection device based on generative perturbation shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0092] like Figure 5As shown, the defect detection device based on generative perturbation may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the defect detection device based on generative perturbation. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the generative perturbation-based defect detection equipment to communicate wirelessly or wiredly with other devices to exchange data. Although generative perturbation-based defect detection equipment with various systems is shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0093] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0094] The defect detection device based on generative perturbation provided in this application, employing the defect detection method based on generative perturbation in the above embodiments, can solve the technical problem of insufficient interpretability in the defect detection model judgment process in related technologies. Compared with the prior art, the beneficial effects of the defect detection device based on generative perturbation provided in this application are the same as those of the defect detection method based on generative perturbation provided in the above embodiments, and other technical features in this defect detection device based on generative perturbation are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0095] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0096] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0097] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the defect detection method based on generative perturbation in the above embodiments.

[0098] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0099] The aforementioned computer-readable storage medium may be included in a defect detection device based on generative perturbation; or it may exist independently and not assembled into a defect detection device based on generative perturbation.

[0100] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the defect detection device based on generative perturbation, the defect detection device based on generative perturbation: acquires an image of the workpiece to be tested, detects the image of the workpiece to be tested according to a pre-trained defect detection model, locates the defect area, and outputs the initial inspection result. Extract the physical attribute vector of the defect region, and continuously perturb the physical attributes of the defect region using the physical attribute vector as the generation condition to generate a variant image sequence with continuously varying defect degree. The variant image sequence is input into the defect detection model for secondary detection to determine the boundary attribute state that causes the detection result of the defect detection model to be flipped. The decision boundary of the defect detection model is calibrated based on the boundary attribute state, and the final detection result of the defect region is output based on the calibrated defect detection model.

[0101] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0103] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0104] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described defect detection method based on generative perturbation. This addresses the technical problem of insufficient interpretability in the defect detection model's judgment process in related technologies. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the defect detection method based on generative perturbation provided in the above embodiments, and will not be repeated here.

[0105] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the defect detection method based on generative perturbation as described above.

[0106] The computer program product provided in this application can solve the technical problem of insufficient interpretability in the defect detection model judgment process in related technologies. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the defect detection method based on generative perturbation provided in the above embodiments, and will not be repeated here.

[0107] All acquisition of signals, information, or actions in this application are carried out in compliance with the relevant data protection laws and policies of the country where the application is located, and with the authorization of the relevant device owner.

[0108] The above description is only a part of the embodiments of this application and does not limit the scope of protection of this application. All equivalent structural transformations made under the technical concept of this application and using the content of this application specification and drawings, or direct / indirect applications in other related technical fields, are included in the scope of protection of this application.

Claims

1. A defect detection method based on generative perturbation, characterized in that, include: Acquire an image of the workpiece to be tested, perform detection on the image of the workpiece to be tested according to a pre-trained defect detection model, locate the defect area, and output the initial inspection result; Extract the physical attribute vector of the defect region, and continuously perturb the physical attributes of the defect region using the physical attribute vector as the generation condition to generate a variant image sequence with continuously varying defect degree. The variant image sequence is input into the defect detection model for secondary detection to determine the boundary attribute state that causes the detection result of the defect detection model to be flipped. The decision boundary of the defect detection model is calibrated based on the boundary attribute state, and the final detection result of the defect region is output based on the calibrated defect detection model.

2. The method as described in claim 1, characterized in that, The step of extracting the physical attribute vector of the defect region includes: Based on the bounding box information of the defect region obtained by the defect detection model, a local defect image is extracted from the image of the workpiece to be tested; The local defect image is input into a physical attribute extraction network, which outputs physical attribute values ​​in multiple dimensions, including at least length, width, and contrast parameters. The physical attribute values ​​of the multiple dimensions are combined into the physical attribute vector; The physical attribute vector is input into the attribute encoder, which maps the physical attribute vector to the latent feature space and generates a set of mutually orthogonal control components, such that adjusting any one of the control components does not affect the other control components.

3. The method as described in claim 1, characterized in that, The step of continuously perturbing the physical properties of the defect region using the physical property vector as the generation condition to generate a variant image sequence with continuously varying defect severity includes: Determine the target attribute parameter to be perturbed from the physical attribute vector; Starting with the current value of the target attribute parameter, multiple decreasing intermediate target values ​​are generated sequentially according to a preset step size, and the starting value and all the intermediate target values ​​constitute a target attribute value sequence. For each target attribute value in the target attribute value sequence, conditional input data is constructed, which includes the current target attribute value, the binary mask image of the defect region, and the background structure guiding feature map. The conditional input data are sequentially input into the conditional diffusion generation model, which generates variant images corresponding to the target attribute values ​​at the latent space feature level. The pixel values ​​outside the defect region in each variant image are consistent with the corresponding pixel values ​​in the image of the workpiece to be tested. The variant images are arranged in descending order of the target attribute value sequence to obtain the variant image sequence.

4. The method as described in claim 1, characterized in that, The step of inputting the variant image sequence into the defect detection model for secondary detection and determining the boundary attribute state that causes the detection result of the defect detection model to undergo class flipping includes: Each variant image in the variant image sequence is sequentially input into the defect detection model for detection, and the defect category confidence and category determination result corresponding to each variant image are obtained; Compare the category determination results of adjacent variant images. When the category determination result of the earlier variant image is a defective category and the category determination result of the later variant image is a good product category, the interval defined by the target attribute value corresponding to the earlier variant image and the target attribute value corresponding to the later variant image is determined as the decision flip interval. Using the decision reversal interval as the initial search interval, the midpoint value of the initial search interval is taken as the detection attribute value; The conditional diffusion generation model is invoked to generate a probe variant image corresponding to the probe attribute value, and the probe variant image is input into the defect detection model for detection to obtain a detection judgment result; If the detection result is a defect category, the lower limit of the initial search interval is updated to the detection attribute value; if the detection result is a good product category, the upper limit of the initial search interval is updated to the detection attribute value. Repeat the operations of taking the midpoint value, generating the probe variant image, and updating the upper and lower limits of the updated search interval until the length of the updated search interval is less than the preset accuracy threshold. The midpoint value of the final search interval is determined as the boundary attribute state.

5. The method as described in claim 4, characterized in that, The steps of calibrating the decision boundary of the defect detection model based on the boundary attribute state, and outputting the final detection result of the defect region based on the calibrated defect detection model include: Calculate the difference between the original value of the target attribute parameter and the state value of the boundary attribute, and use the ratio of the difference to the original value as the defect severity margin; Based on the target attribute value and defect category confidence level corresponding to each variant image in the variant image sequence, an attribute decay curve is generated, and the position of the boundary attribute state is marked in the attribute decay curve; The defect type label, the defect category confidence score, and the defect severity margin output by the defect detection model are combined and output as the final detection result of the defect region.

6. The method as described in claim 5, characterized in that, The step of calibrating the decision boundary of the defect detection model based on the boundary attribute state further includes: A sliding adjustment control associated with the target attribute parameter is provided. In response to the user's drag operation on the sliding adjustment control, the current attenuation ratio is obtained and converted into an instantaneous target attribute value. The conditional diffusion generation model is invoked to generate an instantaneous variant image corresponding to the instantaneous target attribute value, and the instantaneous variant image and the real-time detection results of the defect detection model on the instantaneous variant image are displayed. While displaying the instantaneous variant image and the real-time detection results, the system receives user-submitted category judgment opinions through the feedback operation area. Collect multiple category judgment opinions and corresponding real-time target attribute values ​​received by the feedback operation area to form a feedback sample set; When the number of records in the feedback sample set reaches a preset threshold, the distribution of opinions indicating good products and opinions indicating defects in the feedback sample set is statistically analyzed. Adjust the decision confidence threshold in the defect detection model, which is used to distinguish between defect categories and good product categories, based on the distribution, so that the decision boundary of the defect detection model is calibrated with the user's judgment.

7. A defect detection device based on generative perturbation, characterized in that, The device includes: The acquisition module is used to acquire an image of the workpiece to be tested, detect the image of the workpiece to be tested according to a pre-trained defect detection model, locate the defect area, and output the initial inspection result. The generation module is used to extract the physical attribute vector of the defect region, and use the physical attribute vector as the generation condition to continuously perturb the physical attributes of the defect region to generate a variant image sequence with continuously varying defect degree. The determination module is used to input the variant image sequence into the defect detection model for secondary detection and determine the boundary attribute state that causes the detection result of the defect detection model to be flipped. The detection module is used to perform decision boundary calibration on the defect detection model based on the boundary attribute state, and output the final detection result of the defect area based on the calibrated defect detection model.

8. A defect detection device based on generative perturbation, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the defect detection method based on generative perturbation as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the defect detection method based on generative perturbation as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the defect detection method based on generative perturbation as described in any one of claims 1 to 6.

Images