Generative industrial defect augmentation and detection co-method

Abstract

The application discloses a kind of based on generative industrial defect augmentation and detection collaborative method, S10, real defect sample is input to collaborative optimizer, as initial training data;S20, collaborative optimizer sends sample into target detector and trains;S30, target detector in training process, output its prediction result to input sample, and calculate a feedback signal;S40, intelligent generator receives the random noise vector and feedback signal output by collaborative optimizer, and directional generation new defect sample;S50, generated sample is sent back to collaborative optimizer, and constitutes the data set of next round training;S60, repeat steps S20-S50 until detector performance converges or reaches preset iteration number.The application improves the generalization ability of detector on complex and scarce defect sample through collaborative training, so as to significantly improve the final performance, robustness and sample efficiency of industrial defect detection model under the condition of limited real sample.

Description

Technical Field

[0001] This invention belongs to the field of industrial defect detection technology, and specifically relates to a generative industrial defect augmentation and detection synergy method. Background Technology

[0002] In the grand context of intelligent manufacturing, surface defect detection based on machine vision has become a key technological link in ensuring product quality and improving the level of production automation. Its applications cover a wide range of fields, from high-precision semiconductor wafers and display panels to complex automotive parts and aerospace components. With the rapid development of deep learning, especially Convolutional Neural Networks (CNNs) and Transformer architectures, deep learning-based defect detection models have significantly surpassed traditional image processing methods based on manual features in terms of accuracy and robustness, becoming the mainstream paradigm in the current field of industrial defect detection.

[0003] However, the superior performance of deep learning models is highly dependent on large-scale, high-quality, and evenly distributed labeled datasets. In real-world industrial production environments, constructing such datasets faces three major challenges. First, the scarcity and imbalance of defect samples: Industrial products typically strive for "zero-defect" production, resulting in a far smaller number of defect images collected from actual production lines compared to good product images. Furthermore, the probability of occurrence for various defects (such as scratches, dirt, bumps, dents, and cracks) is extremely uneven; some rare but fatal defect types may have only a handful of samples, forming a typical "long-tail distribution." This data imbalance severely leads to overfitting of the trained detection model to the majority class (good products), while exhibiting extremely poor detection performance for the minority class (especially rare defects), resulting in low recall. Second, the diversity and complexity of defect morphologies: The visual morphologies of industrial defects are highly varied. Not only do defects of the same type (such as scratches) differ greatly in length, width, curvature, direction, and grayscale contrast, but different types of defects may also overlap and become confused in their morphology. Limited real-world samples cannot cover this infinite space of morphological changes, resulting in insufficient model generalization ability and unstable performance when faced with new morphological defects appearing on the production line. Thirdly, the high cost and subjectivity of data annotation: the annotation of defects in industrial images needs to be completed by experienced quality inspection experts, which is not only time-consuming, labor-intensive, and costly, but also may have slight differences in annotation standards among different experts, introducing annotation noise and further limiting the upper limit of model performance.

[0004] To address these challenges, data augmentation techniques have emerged, aiming to generate diverse training samples from limited raw data. Traditional data augmentation methods, such as geometric transformations (rotation, flipping, cropping, scaling), color space transformations (brightness and contrast adjustment), and noise injection, while simple and effective, are limited in the diversity of generated samples by preset transformation rules. They struggle to create novel, highly realistic defect patterns, thus limiting their ability to improve model generalization. In recent years, generative models, represented by Generative Adversarial Networks (GANs) and Diffusion Models, have opened up new avenues for data augmentation due to their powerful data distribution learning and realistic image generation capabilities, becoming a research hotspot in this field.

[0005] The existing implementations most similar to this invention can be mainly divided into two categories: defect sample generation methods based on traditional generative adversarial networks, and sample selection and augmentation methods based on meta-learning or active learning.

[0006] Option 1: Defect Sample Augmentation Method Based on Traditional Generative Adversarial Networks The core idea of ​​this approach is to utilize the generator of a GAN to learn the latent data distribution of real defect samples and sample from it to generate new, unique defect images to expand the training set. A typical implementation process is as follows: First, a limited number of real defect images are collected as training data. Then, a generator G and a discriminator D are constructed. The generator G receives a dataset from a standard normal distribution... Mid-sampled random noise vector As input, the goal is to output a fake defective image. The discriminator D is trained to distinguish whether the input image comes from a real defect sample set. Or a fake sample generated by the generator The entire network training process is a minimax game, and its objective function can be expressed as: ; in, Expressing expectations, It is the data distribution of real defect samples. The prior distribution is noise (usually a Gaussian distribution). By alternately optimizing G and D, the generator G is ultimately expected to capture the true data distribution. ,Right now Once training is complete, new defect samples can be generated indefinitely by inputting random noise into G to expand the original training set, and then used to train the downstream defect detector. .

[0007] However, this method has a limitation: the generator's training objective is merely to "deceive" the discriminator into generating the most "realistic" images possible, completely ignoring the effectiveness of the generated samples for downstream detection tasks. In other words, it may generate a large number of visually realistic samples that offer little help in improving the detector's generalization ability, such as those samples that are already easily recognized by the detector and located inside the decision boundary. This leads to inefficient augmentation, generating a large amount of "redundant" data, but failing to effectively fill in the "difficult" sample regions near the detector's decision boundary.

[0008] Option 2: Sample selection and augmentation methods based on meta-learning or active learning To overcome the blindness of traditional GAN ​​augmentation, some improved schemes attempt to incorporate feedback from downstream detectors into the sample generation or selection process, similar in concept to active learning and meta-learning. For example, a typical implementation employs an iterative loop of "data augmentation-detector training." In each iteration, a detector is first trained using the current dataset. Then, the detector is used to evaluate candidate unlabeled images or images generated by the generative model, identifying those samples that are "most informative." The measure of informativeness is typically the uncertainty of the detector's output; for example, for classification tasks, the entropy of the predicted probabilities can be used. ; in, It is the number of defect categories (including background / good products). The detector will take the image Predicted as category The higher the entropy value, the more uncertain the detector's prediction of the sample, and the more likely the sample is to be near the decision boundary, thus being more valuable for improving model performance.

[0009] After identifying these high-information samples, the scheme adds them (or fine-tunes the generator based on them) to the training set and retrains the detector. This process is repeated cyclically, aiming to achieve optimal detection performance with the fewest augmented samples. However, such methods suffer from a disconnect in the generation phase: the generator (if present) is typically independent and cannot "direct" the generation of the most needed high-information samples based on detector feedback. The generator is either fixed (e.g., using a pre-trained GAN), with a static distribution of generated samples, or its optimization objective is independent of the detector's needs. Therefore, generation and selection are two separate steps, failing to form a closed-loop, mutually reinforcing, collaborative optimization system.

[0010] Based on the above-mentioned existing technical solutions, the following three main drawbacks can be summarized, and the purpose of this invention is to overcome these drawbacks.

[0011] Disadvantage 1: The generation and detection processes are disconnected, resulting in a lack of targeted augmentation. In traditional GAN ​​augmentation schemes, the generator's optimization objective (minimizing the difference between the generated distribution and the real distribution) and the detector's optimization objective (minimizing classification or localization errors) are independent. The generator does not know which types of samples are "scarce" or "critical" for improving detector performance. Therefore, a large number of generated samples may simply be "imitations" of existing samples, failing to effectively expand the detector's decision boundary, leading to inefficient augmentation, high generation costs, and limited performance improvement. In meta-learning or active learning schemes, although feedback from the detector is introduced to select samples, the generation process itself is not "guided" by this feedback. The generator cannot proactively create the "difficult samples" most needed by the detector, and its potential is not fully explored.

[0012] Disadvantage 2: Difficulty in consistently generating "hard" samples, especially adversarial samples. The weakness of a detector often lies near its decision boundary. Samples that can cause the detector to misclassify or have high uncertainty—these "hard samples"—are crucial for improving the model's robustness and generalization ability. In existing technologies, neither static generators nor independent selection mechanisms can stably and controllably generate these highly confusing samples located on the decision boundary. They either generate a large number of "easy" samples or select from a fixed pool, failing to dynamically and purposefully "create" new challenges to "train" the detector.

[0013] Disadvantage 3: Lack of a unified, end-to-end collaborative optimization framework. Existing methods treat data augmentation and detector training as two independent tasks, chained together through simple data transfer or iterative loops. This prevents the entire system from achieving deep coupling and co-evolution between the generator and detector. An ideal system should "learn and build": weaknesses exposed by the detector during training should be fed back to the generator in real time, guiding it to generate samples that compensate for these weaknesses; conversely, more challenging samples generated by the generator should force the detector to learn more robust features, forming a virtuous cycle. Current technologies lack an effective mechanism to achieve this closed-loop collaborative optimization. Summary of the Invention

[0014] To address the shortcomings of existing technologies, this paper proposes a generative model-based method for intelligent augmentation and detection of defects in industrial images, promoting collaborative optimization. This method aims to construct an end-to-end collaborative optimization framework where an intelligent generator (or "agent") generates the most difficult-to-classify defect samples (i.e., adversarial samples) based on real-time feedback from the current detector. Through adversarial collaborative training between the generator and the detector, the generalization ability of the detector on complex and scarce defect samples is continuously and dynamically improved. This significantly enhances the final performance, robustness, and sample efficiency of the industrial defect detection model under limited real-world sample conditions.

[0015] To achieve the above objectives, the technical solution of the present invention is as follows: a generative industrial defect augmentation and detection synergy method, comprising the following steps: S10, real defect samples are input into the co-optimizer as initial training data; S20, the co-optimizer feeds real samples and initially generated samples into the target detector for training; S30, During the training process, the target detector outputs its prediction results for the input samples and calculates a feedback signal; S40, the intelligent generator receives the random noise vector output by the co-optimizer. And feedback signals, to generate new defect samples in a targeted manner; S50, the generated samples are sent back to the co-optimizer and mixed with real samples to form the dataset for the next round of training; S60, repeat steps S20-S50 until the target detector performance converges or the preset number of iterations is reached.

[0016] Preferably, the collaborative optimizer in S10 defines the loss function and training strategy for the entire optimization process. Its core is to construct an end-to-end adversarial collaborative loss, so that the generator and detector promote each other in the adversarial process.

[0017] Preferably, a target detector is defined in S20. The parameters are For an input image The object detector outputs a set of predictions, including candidate regions or object boxes and their corresponding class probability distributions; Focusing on classification tasks, and assuming the detector's output is a... dimensional class probability vector ,in ; We define classification entropy as the core feedback signal, denoted as . : ; The entropy value The range of values ​​is The higher the entropy value, the more uncertain the target detector is in predicting the category of the image, meaning the sample is closer to the detector's decision boundary and the more difficult it is for the current target detector; conversely, the lower the entropy value, the more confident the target detector is in its category judgment, and the sample is a simple sample.

[0018] Preferably, the intelligent generator in S40 is a conditional generation model, and its generation process can be represented as follows: ; in, These are the parameters of the generator. To achieve this conditional generation, the generator's network structure is based on a conditional GAN ​​or conditional diffusion model; within the conditional GAN ​​framework, the generator... Receive a concatenated condition vector As input, its corresponding authenticity discriminator It is responsible for distinguishing between real and generated images, and ensuring that the generated images meet the given conditions. .

[0019] Preferably, in S30, the parameters of the generator are frozen by the co-optimizer. and using a real sample set and the hard sample set generated by the current generator The mixed dataset is used to update the object detector. The training objective of the object detector is to minimize its prediction error across all samples, and its loss function is... It consists of two parts: the standard supervised loss on real samples. and the loss associated with generating samples. To encourage the detector to learn from the generated difficult samples, the fit to these samples is: ; in, It is classification loss. It is a hyperparameter used to balance the importance of real samples and generated samples. These are the labels of the real samples; for the generated samples... Its label It is known that by minimizing The object detector is forced to learn the features of difficult samples created by the intelligent generator that are located near its current decision boundary, thereby pushing its decision boundary to move in a more robust direction.

[0020] Preferably, in step S40, the co-optimizer freezes the parameters of the target detector. The intelligent generator is updated based on the latest feedback signal from the object detector. The training objective of the intelligent generator is no longer simply to generate realistic images, but to generate images that maximize the uncertainty of the object detector while maintaining semantic authenticity, thus constituting an adversarial objective. The loss function of the intelligent generator is defined. for: ; In this loss function: The first term: the adversarial entropy maximization term, which is the core of collaborative optimization, in which... It is a measure of the uncertainty of the current object detector in generating images; the intelligent generator minimizes this uncertainty. To maximize That is, attempting to generate images that will most confuse the detector and create the highest uncertainty, with coefficients... Controlling the strength of this term directly drives the intelligent generator to generate samples in a direction that aligns with the decision boundary of the target detector; The second item is the realism constraint, which ensures that the generated images are visually realistic and conform to the visual characteristics of industrial defects. It is a realism loss function, derived from a pre-trained or synchronously trained realism discriminator. This is provided to prevent intelligent generators from generating meaningless, unrealistic noise in order to maximize the uncertainty of the target detector. The specific formula is as follows: ; This is crucial for the intelligent generator to maintain image quality; The third constraint is the controllability of difficulty, ensuring that the intelligent generator can not only generate samples with high uncertainty, but also adjust the difficulty level according to the input. To achieve precise control; It is a conditional consistency loss that measures the actual entropy value produced by the generated image after passing through the object detector. With target entropy The difference between them is defined as mean square error: ; By introducing this element, the intelligent generator is trained into a dial-controllable tool: by setting different... Values ​​are generated to produce defect samples ranging from simple to extremely difficult, with continuously controllable difficulty. During training, coefficients To balance these three losses, by minimizing The intelligent generator, while maintaining realism, learns how to generate the most difficult and challenging samples from the perspective of the current object detector.

[0021] The present invention has at least the following beneficial effects: 1. Achieved deep collaboration and closed-loop optimization between generation and detection: This invention constructs a closed-loop system through a collaborative optimizer, where the generator and detector mutually feed back, compete, and co-evolve. The performance feedback from the detector directly becomes the generator's optimization objective (maximizing entropy), guiding it to generate "difficult samples." These "difficult samples" then become key data for the detector's next round of training, forcing the detector to improve its discrimination ability. This deeply coupled mechanism fundamentally solves the problem of the separation between generation and detection in existing technologies, achieving a collaborative evolution of "learning and building simultaneously, with learning and building complementing each other."

[0022] 2. Significantly improves the targeting and efficiency of sample augmentation: Since the generator aims to maximize the uncertainty of the detector, the generated samples naturally focus on the weakest and most learning-intensive regions of the current detector—namely, the vicinity of the decision boundary. Unlike the "random" samples generated by traditional GANs or the "existing" samples selected through active learning, this invention generates "customized" adversarial samples specifically designed to "attack" the weaknesses of the current detector. This ensures that the data augmented in each round has maximized marginal utility for improving detector performance, greatly improving the efficiency of data augmentation and achieving superior model performance with limited computational resources and training time.

[0023] 3. Achieves fine-grained control over the difficulty of generated samples: This invention introduces a difficulty controllability constraint term into the generator's loss function. This allows the generator to adjust the input based on the target difficulty level. Generate samples of varying difficulty. This provides significant flexibility for subsequent training strategies. For example, a "curricular learning" strategy can be implemented, requiring the generator to produce samples of moderate difficulty in the early stages of training to help the detector establish basic discriminative capabilities; in the later stages of training, the target difficulty can be gradually increased. By generating extremely difficult samples, the detector is "reinforced" to systematically and progressively improve the model's generalization ability.

[0024] This enhances the detector's ability to generalize to rare and complex defects: through continuous adversarial training, the detector is constantly exposed to novel and challenging defect morphologies "generated" by the generator and located near its decision boundary. This adversarial training forces the detector to learn more robust and discriminative feature representations, rather than simply memorizing the limited patterns in the training set. Therefore, the ultimately trained detector exhibits stronger generalization ability and higher detection robustness for real-world defects with varied morphologies not seen in the training set, and in particular, significantly improves recall for rare defects (the tail of a long-tailed distribution). Attached Figure Description

[0025] Figure 1 This is a flowchart of the steps of the generative industrial defect augmentation and detection synergy method of the present invention; Figure 2 This is a schematic diagram of the S50 process of the present invention based on the generative industrial defect augmentation and detection synergy method. Detailed Implementation

[0026] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0027] See Figure 1 The flowchart below illustrates the steps of an embodiment of the present invention, including the following steps: S10, real defect samples are input into the co-optimizer as initial training data; S20, the co-optimizer feeds real samples and initially generated samples into the target detector for training; S30, During the training process, the target detector outputs its prediction results for the input samples and calculates a feedback signal; S40, the intelligent generator receives the random noise vector output by the co-optimizer. And feedback signals, to generate new defect samples in a targeted manner; S50, the generated samples are sent back to the co-optimizer and mixed with real samples to form the dataset for the next round of training; S60, repeat steps S20-S50 until the target detector performance converges or the preset number of iterations is reached.

[0028] Target detector The detector is the final output of the invention's performance, and its structure can be any advanced detection network, such as Faster R-CNN or YOLO series based on CNN, or DETR based on Transformer. In this invention, the detector is not merely a trained object, but also a source of feedback signals. Its core functions are twofold: 1) to detect defects in the input image and output the detection results; 2) to calculate a feedback signal that quantifies the "difficulty" of each input sample based on its current state. This feedback signal serves as "navigation information" guiding the intelligent generator's generation direction.

[0029] Define target detector The parameters are For an input image The detector outputs a set of predictions, including candidate regions (or bounding boxes) and their corresponding class probability distributions. For simplicity, we focus on a classification task and assume the detector's output is a... dimensional class probability vector ,in In this invention, we define classification entropy as the core feedback signal, denoted as . : ; The entropy value The range of values ​​is A higher entropy value indicates that the detector is less certain about the category prediction of the image, meaning the sample is closer to the detector's decision boundary and is more "difficult" for the current detector. Conversely, a lower entropy value indicates that the detector is very confident in its category judgment, and the sample is a "simple" sample.

[0030] Intelligent Generator This is the core of the invention for achieving targeted augmentation. Unlike ordinary GAN generators, it not only receives random noise... It also receives feedback signals from the detector. This serves as a conditional input, enabling precise control over the "difficulty" of the generated samples. Its goal is to achieve this at a given difficulty level. ( yes Under a quantized value, an image is generated that causes the detector to produce a corresponding uncertainty. Therefore, the intelligent generator is essentially a conditional generation model, and its generation process can be represented as: ; in, These are the parameters of the generator. To achieve this conditional generation, the generator's network structure can be based on a conditional GAN ​​or a conditional diffusion model. For example, within the conditional GAN ​​framework, the generator... Receive a concatenated condition vector As input. Its corresponding discriminator (to maintain distinction from the detector, we call it the authenticity discriminator). This is responsible for distinguishing between real and generated images, and ensuring that the generated images meet the given conditions. The training objective of the intelligent generator is defined in the subsequent co-optimizer.

[0031] Collaborative optimizer It is the "brain" that connects the generator and detector and drives the co-evolution of the entire system. It defines the loss function and training strategy for the entire optimization process. Its core is to build an end-to-end adversarial co-optimization loss, which enables the generator and detector to promote each other in adversarial competition.

[0032] The training process of the collaborative optimizer is carried out alternately, with each iteration round... It includes the following two main steps: S20-S30 detector training (Detector Update) In this step, the co-optimizer freezes the generator's parameters. and using a real sample set and the "high difficulty" sample set generated by the current generator The detector is updated using a mixed dataset. The training objective of the detector is to minimize its prediction error across all samples. Its loss function... It consists of two parts: the standard supervised loss on real samples. and the loss associated with generating samples. To encourage the detector to learn from the generated difficult samples, we place particular emphasis on fitting these samples.

[0033] ; in, It is a classification loss (such as cross-entropy loss). It is a hyperparameter used to balance the importance of real samples and generated samples. These are the labels of the real samples. For the generated samples... Its label This is known (because we know what type of defect the generator is trying to generate), and is usually the generator's target defect category. By minimizing The detector is forced to learn the features of difficult samples created by the generator that are near its current decision boundary, thereby pushing its decision boundary toward a more robust direction.

[0034] S40-S50 Intelligent Generator Training (Generator Update) In this step, the co-optimizer freezes the detector's parameters. The generator is updated based on the latest feedback signal from the detector. The training objective of the generator is no longer simply to generate realistic images, but to generate images that maximize the detector's uncertainty while maintaining the semantic realism of the images (i.e., the generated images must look like real defects, not meaningless noise). This constitutes an adversarial objective. We define the generator's loss function... for: ; We will explain this loss function in detail: The first term: the term maximizing adversarial entropy. This term is the core of the collaborative optimization in this invention. Among them, It is the uncertainty measure (entropy) of the current detector on the generated image. The generator minimizes... To maximize This involves attempting to generate images that will "most confuse" the detector and introduce the highest uncertainty. (Coefficients) This controls the strength of the term. This term directly drives the generator to generate samples oriented towards the detector's decision boundary.

[0035] The second item: Realism constraint. This item is used to ensure that the generated images are visually realistic and conform to the visual characteristics of industrial defects. It is a realism loss function, which can be derived from a pre-trained or realism discriminator trained in sync with the generator. This can be provided, for example, using the Wasserstein distance in a Wasserstein GAN or the adversarial loss of a standard GAN. The aim is to prevent the generator from producing meaningless, unrealistic noise to maximize detector uncertainty. A common form is: ; If WGAN-GP is used, the corresponding method is... This is crucial for the generator to maintain image quality.

[0036] The third item: Difficulty controllability constraint. This item is an important innovation, ensuring that the generator can not only generate samples with high uncertainty, but also adjust the difficulty level according to the input. To carry out precise control. It is a conditional consistency loss that measures the actual entropy value produced by the generated image after passing through the detector. With target entropy The difference between them. For example, we can define it as mean squared error: ; By introducing this element, the generator is trained into a "dial" controllable tool: we can control it by setting different... Values ​​(e.g., from low to high) are used to generate defect samples with continuously controllable difficulty, ranging from "easy" to "extremely difficult". This is of great value in subsequent detector training strategies, such as enabling curriculum learning from easy to difficult.

[0037] During training, coefficients This is used to balance these three losses. By minimizing... While maintaining realism, the generator has learned how to generate the most difficult and challenging samples from the perspective of the current detector.

[0038] The entire collaborative optimization process is an iterative, alternating adversarial training process, as shown below: Collaborative optimization iterative process 1. Initialization: Use a small number of real defect samples Pre-trained detectors and generator (The generator can be initialized using the traditional GAN ​​method first).

[0039] 2. For iteration rounds to do: / / Phase 1: Generating Hard Samples **For** \( k = 1 \) **to** \( K \) **do**: Sample a batch of noise vectors \( \{\mathbf{z}_i\} \) and target difficulty level \( \{s_i\} \). Generate a sample set using the current generator \( \mathcal{G}_{intel} \) \( \mathcal{D}_{gen} \leftarrow \{\mathcal{G}_{intel}(\mathbf{z}_i, s_i)\} \).

[0040] **End For** / / Phase Two: Training the Detector Construct a mixed training set \( \mathcal{D}_{mix} = \mathcal{D}_{real} \cup \mathcal{D}_{gen} \). Use gradient descent to update the detector parameters \( \theta_{det} \leftarrow \theta_{det} - \eta_{det} \nabla_{\theta_{det}} L_{det}(\theta_{det}) \) according to formula (4). / / Stage 3: Training the intelligent generator **For** \( m = 1 \) **to** \( M \) **do**: Sample a batch of noise vectors \( \{\mathbf{z}_j\} \) and target difficulty level \( \{s_j\} \). The generator loss \( ​​L_{gen} \) is calculated according to formula (5), where the detector is fixed.

[0041] Update the generator parameters using gradient descent: \( \theta_{gen} \leftarrow \theta_{gen}- \eta_{gen} \nabla_{\theta_{gen}} L_{gen}(\theta_{gen}) \).

[0042] **End For** 3. End For 4. Output: A high-performance detector after training. .

[0043] The co-optimizer, acting as the core control unit, receives initial data from a real defect sample library. In each iteration, the co-optimizer feeds the current training set (containing real samples and samples generated in the previous round) into the target detector for training. During training, the target detector calculates an uncertainty metric, i.e., a feedback signal, for its input samples. This data is then fed back to the co-optimizer. The co-optimizer processes the random noise vector... and the expected level of difficulty As a condition, the intelligent generator is driven by feedback signals. Targeted generation to produce new difficult samples These new samples are added to the training set for the next round of detector training. This process is repeated, forming a closed loop, until the detector performance converges.

[0044] See Figure 2 This is a specific implementation structure of the intelligent generator in S50 of the present invention (taking a conditional generative adversarial network as an example). As shown in the figure, the generator... Receive random noise vector and the target difficulty level passed from the co-optimizer As input conditions, after a series of nonlinear transformations, the generated defect image is output. The image is then fed into a realism discriminator. To ensure its visual realism (corresponding to the loss term) On the other hand, it is sent to the current target detector. The detector outputs a measure of the uncertainty of the image. The co-optimizer is based on and target difficulty The difference is calculated by maximizing the adversarial entropy and the difficulty controllability constraint, and combined with the realism loss to form the generator's total loss. The generator parameters are guided by backpropagation. The updates enable it to learn to generate defect samples that are both realistic and meet specific difficulty requirements.

[0045] The key points and protection points of this invention lie in the following interrelated core technical features, which together constitute the substantial characteristics and progress of this invention that distinguish it from the prior art: 1. Directional generation mechanism based on detector feedback signal: the core lies in the intelligent generator. In the optimization objective, a maximization of the target detector is introduced. Uncertainty in output (such as classification entropy) The core adversarial loss term (the first term in Formula 5) is the "command stick" for the entire collaborative optimization, ensuring that the generator's generation process is not blind, but purposefully approaches the decision boundary of the detector, creating "difficult samples" in a targeted manner. The key lies in the technical idea of ​​using the detector's real-time state (such as the predicted probability distribution) as the generator's optimization signal and its specific implementation.

[0046] 2. Techniques to control the difficulty of generating samples: By introducing a conditional consistency constraint term into the generator loss function. This allows the generator to adjust the target difficulty level based on the external input. Samples with corresponding uncertainties are generated. This not only demonstrates the generator's capabilities but also provides flexible training scheduling capabilities for the entire collaborative optimization framework. The safeguard lies in this technical solution, which treats the "difficulty" of generating samples as an adjustable conditional variable and uses the detector's uncertainty metric for closed-loop calibration.

[0047] 3. An alternating adversarial cooperative optimization framework for generators and detectors: Cooperative Optimizer The defined end-to-end, alternating optimization training process (Algorithm 1), particularly the strategy of alternating updates to the detector and generator, utilizing each other's latest state in each update, constitutes a complete closed-loop learning system and is the macroscopic architecture for achieving "co-evolution" in this invention. The protection lies in this complete iterative process encompassing "detector training - generator training based on detector feedback - retraining the detector."

[0048] 4. Training objective of the intelligent generator integrating multiple loss functions: The intelligent generator proposed in this invention integrates multiple loss functions. The specific combination of these three terms—adversarial entropy maximization, realism constraint, and difficulty controllability constraint—ensures that the generated samples are both "difficult," "realistic," and "controllable," forming the cornerstone of the beneficial effects of this invention. The protection lies in the design of this three-term combination loss function and its application in collaborative optimization.

[0049] The core idea of ​​this invention, cooperative directional augmentation based on detector feedback, can be achieved through various alternative or variant schemes, all while still fulfilling the objectives of this invention. Several possible alternative schemes are listed below: 1. Alternatives to uncertainty measures: in feedback signals In terms of design, besides classification entropy, other metrics that can quantify detector uncertainty can be used. For example, for bounding box regression-based detectors, the variance of predicted boxes or the statistical variance of multiple predictions obtained from Monte Carlo Dropout can be used as an uncertainty measure. For energy-based models, energy values ​​can be used. Even the consistency of model predictions across multiple training snapshots (i.e., model divergence) can be used as feedback. These metrics can all replace entropy as a target for generator optimization.

[0050] 2. Alternative to the generative model structure: The intelligent generator of this invention This is not limited to conditional GANs. Conditional diffusion models can be used as generators. In diffusion models, the feedback signal can be... As a condition embedded into the denoising network at each step, it generates conditions by reversing the diffusion process. Images. Diffusion models may outperform GANs in certain scenarios due to their generative diversity and training stability. In addition, variational autoencoders (VAEs) and their variants (such as conditional VAEs) can also serve as alternative generators, achieving directional generation through interpolation and conditional control of their latent space.

[0051] 3. Alternatives to Co-optimization Training Strategies: Besides alternating updates, joint updates or meta-learning can be used to update the co-optimizer. For example, the detector and generator can be viewed as two parts of a larger "meta-model," with their parameters updated simultaneously through a single forward-backward propagation, optimizing a meta-objective function that incorporates the detector's final performance. The idea is that the generator not only needs to maximize the uncertainty of the current detector but also needs to anticipate that its generated samples, after updating the detector, will minimize the detector's generalization error. This can be achieved by calculating the second-order gradient (e.g., the MAML algorithm), theoretically potentially yielding stronger co-optimization effects.

[0052] 4. Alternatives to difficulty scheduling strategies: When achieving difficulty controllability, besides directly adding to the loss function... Furthermore, more complex scheduling strategies can be employed. For example, during generator training, instead of explicitly incorporating conditional loss, the target difficulty level can be adjusted. The sampling distribution is used to indirectly control the process. For example, in the early stages of training... The primary sampling originates from low-entropy regions, allowing the generator to initially learn to produce simple samples. As the detector's capabilities improve, the sampling distribution is gradually shifted towards high-entropy regions. This "curriculum sampling" strategy can replace explicit conditional constraints, but its implementation is more indirect.

[0053] 5. Alternatives to Multi-Detector Collaborative Feedback: The feedback signal source can be extended to an integrated feedback from multiple heterogeneous detectors (e.g., different architectures or different initializations). For example, the generator's objective can be set to maximize the divergence of the outputs of multiple detectors, such as the sum of the KL divergences of the predicted class distributions. In this way, the generated samples can simultaneously "attack" the weaknesses of multiple detectors, forcing the final ensemble model or a single model to improve performance in a wider range of "weak regions," thereby achieving stronger robustness and generalization ability.

[0054] The above description of the embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make several improvements and modifications to the present invention without departing from the principles of the present invention, and these improvements and modifications also fall within the protection scope of the claims of the present invention.

[0055] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A generative industrial defect augmentation and detection synergy method, characterized in that, Includes the following steps: S10, real defect samples are input into the co-optimizer as initial training data; S20, the co-optimizer feeds real samples and initially generated samples into the target detector for training; S30, During the training process, the target detector outputs its prediction results for the input samples and calculates a feedback signal; S40, the intelligent generator receives the random noise vector output by the co-optimizer. And feedback signals, to generate new defect samples in a targeted manner; S50, the generated samples are sent back to the co-optimizer and mixed with real samples to form the dataset for the next round of training; S60, repeat steps S20-S50 until the target detector performance converges or the preset number of iterations is reached.

2. The method for generative industrial defect augmentation and detection synergy according to claim 1, characterized in that, The collaborative optimizer in S10 defines the loss function and training strategy for the entire optimization process. Its core is to construct an end-to-end adversarial collaborative loss, which enables the generator and detector to promote each other in adversarial competition.

3. The method for generative industrial defect augmentation and detection synergy according to claim 2, characterized in that, The target detector is defined in S20. The parameters are For an input image The object detector outputs a set of predictions, including candidate regions or object boxes and their corresponding class probability distributions; Focusing on classification tasks, and assuming the detector's output is a... dimensional class probability vector ,in ; We define classification entropy as the core feedback signal, denoted as . : ； The entropy value The range of values ​​is The higher the entropy value, the more uncertain the target detector is in predicting the category of the image, meaning the sample is closer to the detector's decision boundary and the more difficult it is for the current target detector; conversely, the lower the entropy value, the more confident the target detector is in its category judgment, and the sample is a simple sample.

4. The generative industrial defect augmentation and detection synergy method according to claim 3, characterized in that, The intelligent generator in S40 is a conditional generation model, and its generation process can be represented as follows: ； in, These are the parameters of the generator. To achieve this conditional generation, the generator's network structure is based on a conditional GAN ​​or conditional diffusion model; within the conditional GAN ​​framework, the generator... Receive a concatenated condition vector As input, its corresponding authenticity discriminator It is responsible for distinguishing between real and generated images, and ensuring that the generated images meet the given conditions. .

5. The generative industrial defect augmentation and detection synergy method according to claim 4, characterized in that, The parameters of the co-optimizer freeze generator in S30 and using a real sample set and the hard sample set generated by the current generator The mixed dataset is used to update the object detector. The training objective of the object detector is to minimize its prediction error across all samples, and its loss function is... It consists of two parts: the standard supervised loss on real samples. and the loss associated with generating samples. To encourage the detector to learn from the generated difficult samples, the fit to these samples is: ； in, It is classification loss. It is a hyperparameter used to balance the importance of real samples and generated samples. These are the labels of the real samples; for the generated samples... Its label It is known that by minimizing The object detector is forced to learn the features of difficult samples created by the intelligent generator that are located near its current decision boundary, thereby pushing its decision boundary to move in a more robust direction.

6. The generative industrial defect augmentation and detection synergy method according to claim 5, characterized in that, In S40, the co-optimizer freezes the parameters of the target detector. The intelligent generator is updated based on the latest feedback signal from the object detector. The training objective of the intelligent generator is no longer simply to generate realistic images, but to generate images that maximize the uncertainty of the object detector while maintaining semantic authenticity, thus constituting an adversarial objective. The loss function of the intelligent generator is defined. for: ； In this loss function: The first term: the adversarial entropy maximization term, which is the core of collaborative optimization, in which... It is a measure of the uncertainty of the current object detector in generating images; the intelligent generator minimizes this uncertainty. To maximize That is, attempting to generate images that will most confuse the detector and create the highest uncertainty, with coefficients... Controlling the strength of this term directly drives the intelligent generator to generate samples in a direction that aligns with the decision boundary of the target detector; The second item is the realism constraint, which ensures that the generated images are visually realistic and conform to the visual characteristics of industrial defects. It is a realism loss function, derived from a pre-trained or synchronously trained realism discriminator. This is provided to prevent intelligent generators from generating meaningless, unrealistic noise in order to maximize the uncertainty of the target detector. The specific formula is as follows: ； This is crucial for the intelligent generator to maintain image quality; The third constraint is the controllability of difficulty, ensuring that the intelligent generator can not only generate samples with high uncertainty, but also adjust the difficulty level according to the input. To carry out precise control; It is a conditional consistency loss that measures the actual entropy value produced by the generated image after passing through the object detector. With target entropy The difference between them is defined as mean square error: ； By introducing this element, the intelligent generator is trained into a dial-controllable tool: by setting different... Values ​​are generated to produce defect samples ranging from simple to extremely difficult, with continuously controllable difficulty. During training, coefficients To balance these three losses, by minimizing The intelligent generator, while maintaining realism, learns how to generate the most difficult and challenging samples from the perspective of the current object detector.

Images