A method and system for continual learning in the field of defect detection

Abstract

This invention discloses a continuous learning method and system in the field of defect detection, including training and deploying an initial defect detection model; online detection and evaluation of sample value by a monitor, selecting high-value samples for annotation cache; performing human-computer interactive annotation on high-value samples; triggering incremental training when preset conditions are met; incremental fine-tuning of the model using anti-forgetting techniques; deploying an updated model after verification; and iteratively achieving continuous learning. This invention, through a multi-strategy fusion sample value evaluation method, accurately identifies new defects, boundary samples, and uncertain difficult samples in the production line, and performs targeted annotation only on high-value samples, significantly reducing the manpower and time costs of manual annotation, avoiding redundant storage of invalid samples, and effectively solving the problems of low sample utilization and lagging model updates in existing technologies.

Description

Technical Field

[0001] This invention relates to the fields of industrial product quality inspection and machine learning technology, specifically a continuous learning method and system in the field of defect detection. Background Technology

[0002] In modern industrial automated production, surface defect detection is a crucial step in ensuring product quality and production yield. With the development of deep learning technology, defect detection methods based on deep neural networks have been widely applied to the appearance quality inspection of various products. These methods typically involve offline training using a large number of labeled samples, followed by deployment of the trained model to the production line for online inference and detection.

[0003] In real-world industrial applications, the types, shapes, sizes, and distributions of product defects dynamically change due to factors such as adjustments in production processes, changes in raw materials, equipment wear and tear, and environmental variations. New defect types and abnormal defect samples constantly emerge. Traditional defect detection models, once trained on a fixed dataset, struggle to adapt their detection capabilities to changes in the production line, leading to frequent missed and false detections.

[0004] To maintain model detection performance, current technologies typically employ a method of manually and continuously collecting production line images, labeling defects image by image, and periodically retraining the model to update it. This approach has significant drawbacks:

[0005] First, manual data collection and labeling are extremely costly, requiring a large number of quality inspectors to work on-site for extended periods, which consumes a significant amount of manpower and time.

[0006] Second, the vast majority of the images collected are normal samples or typical defective samples that the model has already stably identified, which contribute very little to improving the model's performance and result in low sample utilization efficiency.

[0007] Third, difficult examples that are most valuable for model optimization, such as easily confused defects, defects with ambiguous boundaries, and novel defects, are difficult to capture automatically; fourth, the long model update cycle significantly prolongs the time from equipment debugging to formal production; fifth, when the model is retrained on new data, it is prone to catastrophic forgetting problems, resulting in a decline in the ability to detect the original defect types.

[0008] Currently, there is no existing technology that can form a continuous learning framework suitable for industrial defect detection scenarios, automatically identify high-value samples, reduce annotation costs, achieve incremental updates, and prevent catastrophic forgetting, making it difficult for defect detection models to quickly adapt to dynamic changes in production lines. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of existing technologies, such as high manual annotation costs, low sample utilization efficiency, slow model update speed, difficulty in adapting to dynamic defect changes in production lines, and catastrophic forgetting of models. It provides a continuous learning method and system in the field of defect detection, which automatically identifies high-value samples such as potential missed detections, false detections, boundary defects, and novel defects through a monitoring mechanism, so as to achieve rapid model iteration and update at low annotation costs, enabling the defect detection model to continuously adapt to changes in the production line.

[0010] To achieve the above objectives, the present invention provides the following technical solution:

[0011] A continuous learning method in the field of defect detection includes the steps of training a defect detection model using an initial labeled dataset and deploying the model to the production site to perform online detection, specifically including the following steps:

[0012] Step S1: Train an initial defect detection model using the initial labeled dataset, and deploy the initial defect detection model to the online detection module for real-time defect detection of products on the production line.

[0013] Step S2: The online detection module performs real-time defect detection on the surface image of the product on the production line, outputs relevant information such as defect category, defect location, and detection confidence level, and records the image data and detection results synchronously. The monitoring module evaluates the value of each sample according to the preset strategy, calculates the sample value score, and determines the samples with value scores higher than the preset threshold as high-value samples, and sends the high-value samples to the labeling cache area.

[0014] Step S3: Through the human-computer interaction annotation module, the annotators perform annotation operations such as selecting defect locations and annotating defect categories on high-value samples in the annotation buffer area. After the annotation is completed, the samples are moved into the annotated dataset.

[0015] Step S4: When the number of samples in the labeled dataset reaches a preset threshold or a preset time interval condition is reached, the incremental training process of the defect detection model is automatically triggered.

[0016] Step S5: Using catastrophic forgetting prevention techniques such as Elastic Weight Solidification (EWC) or Knowledge Distillation, the current defect detection model is incrementally fine-tuned and trained using newly labeled data and combined with historical core data. After training, the updated defect detection model is obtained.

[0017] Step S6: Verify the updated defect detection model on the standard test set for indicators such as detection accuracy, false negative rate, and false positive rate. After verification, deploy the updated model to the online detection module to replace the original old model, and clear the unlabeled cache area or retain some samples according to the preset strategy.

[0018] Step S7: Repeat steps S2 to S6 to enable the model to continuously acquire high-value samples and complete iterative optimization, thereby achieving continuous learning of the defect detection model.

[0019] Furthermore, the value assessment strategy executed by the monitoring module includes:

[0020] Uncertainty estimation is based on the softmax probability entropy or maximum class probability output by the model. When the confidence level is lower than a preset threshold, the sample is judged as a high-value sample.

[0021] Boundary sample detection obtains the sample embedding vector through a feature extractor, calculates the nearest neighbor distance between the sample and the existing training set features, and determines the sample as a high-value sample when the distance exceeds a preset threshold.

[0022] Consistency check: After performing data augmentation operations such as rotation and cropping on the same image and inputting it into the model again, if the output results are inconsistent multiple times, it is judged as a high-value sample.

[0023] The autoencoder reconstruction error is determined by reconstructing the input image using an autoencoder trained on normal samples. When the reconstruction error is greater than a preset threshold, the image is considered a high-value sample.

[0024] Predictive drift detection compares the output of the current model with that of historical versions of the model, and identifies samples as high-value samples when the difference in output exceeds a preset range.

[0025] Furthermore, during incremental training, data balancing and replay steps are set up to sample typical defect samples and difficult samples from historical data and mix them with newly labeled data to participate in training, so as to avoid catastrophic forgetting of the model and maintain the ability to detect historical defect types.

[0026] A continuous learning system in the field of defect detection includes:

[0027] An online inspection module deployed on the production line is used to perform real-time inspection of product images using the current version of the defect detection model, outputting defect category, defect location, confidence level, and recording the inspection results.

[0028] The monitoring module is used to evaluate the value of the detected samples, calculate the value score based on at least one of the strategies of uncertainty estimation, boundary sample detection, consistency check, autoencoder reconstruction error, and prediction drift detection, and mark high-value samples.

[0029] The sampling and storage module is used to receive the images and detection results output by the online detection module, store high-value samples in the annotation buffer, and discard or retain a small number of low-value samples.

[0030] The human-computer interaction annotation module provides an interactive interface for annotators to complete the annotation of defect locations and defect categories of high-value samples, forming an annotated dataset.

[0031] The incremental training module is used to incrementally fine-tune the current model by employing anti-forgetting techniques such as elastic weight solidification or knowledge distillation when preset triggering conditions are met, thereby obtaining an updated model.

[0032] The model validation and deployment module is used to validate the accuracy, false negative rate, and false positive rate of the updated model. After the validation meets the standards, the updated model is deployed to the online detection module to replace the old model.

[0033] Preferably, the system may also include a data balancing and replay module, which is used to sample representative samples from historical data and mix them with newly labeled data to participate in incremental training, preventing the model from forgetting historical deficiencies.

[0034] The unlabeled cache area is updated with a first-in-first-out (FIFO) or value score priority strategy. After the model is deployed, the unlabeled cache area is cleared or some samples are retained according to a preset strategy.

[0035] Compared with the prior art, the beneficial effects of the present invention are:

[0036] 1. This invention uses a multi-strategy fusion sample value assessment method to accurately discover new defects, boundary samples, and uncertain and difficult sample samples in the production line. It only performs targeted annotation on high-value samples, which significantly reduces the manpower and time cost of manual annotation, avoids redundant storage of invalid samples, and effectively solves the problems of low sample utilization and lagging model updates in existing technologies.

[0037] 2. In the incremental learning process of the model, this invention combines the replay of historical core data with multiple anti-forgetting technologies to complete incremental fine-tuning. It can quickly learn new defect features while stably retaining the ability to detect historical classic defects, overcoming the catastrophic forgetting defect of the model. Through a closed-loop continuous learning architecture of detection, screening, annotation, training, verification and deployment, it can dynamically adapt to fluctuations in production conditions and maintain the overall accuracy of defect detection for a long time. Attached Figure Description

[0038] Figure 1 This is a diagram of the overall system module architecture of the present invention;

[0039] Figure 2It is the complete business closed-loop flowchart of the present invention;

[0040] Figure 3 It is the breakdown block diagram of the five major sample value evaluation strategies of the present invention. Specific implementation manners

[0041] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0042] Please refer to Figure 1-3 , the present invention provides a technical solution:

[0043] A continuous learning method in the field of defect detection, and its specific implementation steps are as follows:

[0044] Step S1, select an industrial product surface defect public initial labeled data set, which contains both normal defect-free samples and multiple types of typical defect samples at the same time. Select YOLOv5 as the basic defect detection network model, and complete offline training relying on the conventional supervised learning training paradigm. After convergence, obtain a solidified initial defect detection model; deploy the trained initial defect detection model to the online detection module supporting the production line as the basic inference model for real-time defect recognition on the production line.

[0045] Step S2, the online detection module continuously collects the surface images of the products on the production line with an industrial high-definition camera, and uniformly performs preprocessing with a fixed specification on the original images, including uniformly scaling the size to the model input size, Gaussian noise filtering, and gray-scale normalization processing to ensure the uniformity of the input data distribution; the preprocessed images are sent to the current defect detection model to complete forward inference, and stably output the defect category, defect bounding box coordinates, and single-category prediction confidence corresponding to each frame of image, and synchronously store the original image, inference result, and confidence value.

[0046] The monitor module simultaneously conducts multi-dimensional value quantification evaluation on each online collected sample, and runs multiple evaluation logics in parallel.

[0047] Among them, uncertainty estimation is quantified and calculated using the category probability information entropy, and the calculation formula is as follows:

[0048]

[0049] In the formula, represents the information entropy, which is used to characterize the uncertainty of model inference, represents the total number of defect categories, represents the For the probability of defects, the engineering fixed judgment condition is: when the information entropy is greater than 0.35, or the maximum prediction confidence of a single sample is less than 0.60, it is directly judged as an uncertain high-value sample.

[0050] Boundary sample detection is achieved by obtaining sample embedding vectors through a feature extractor and calculating the Euclidean distance between the vectors and the features of the training set. The calculation formula is as follows:

[0051]

[0052] In the formula, It is represented as the Euclidean distance between eigenvectors. These are represented as the feature vectors of the sample to be tested and the training set samples, respectively. It is represented as the feature vector dimension, with a fixed distance threshold set to 1.80. When the distance exceeds the preset threshold, the sample is determined to be a boundary sample or an out-of-distribution sample.

[0053] The consistency check adopts a fixed combination data augmentation strategy, uniformly implementing three fixed augmentation methods: ±15° random rotation, 0.8x cropping of the central region, and horizontal flipping. The original image and the augmented image are fed into the model for inference. If the defect category is inconsistent before and after, or the defect box intersection-union ratio is less than 0.5, it is judged as a high-value sample with unstable prediction.

[0054] An autoencoder trained on normal samples is used to reconstruct the image, and the reconstruction error in the form of mean square error is calculated using the following formula:

[0055]

[0056] In the formula, This is represented as reconstruction error. These are represented as the image width and height, respectively. Represented as raw pixel values, This represents the pixel value reconstructed by the autoencoder. The reconstruction error threshold is fixed at 0.025. When the reconstruction error is greater than the preset threshold, it is judged as an abnormal sample.

[0057] By comparing the probability distributions output by the current model with those of historical versions, the prediction drift is calculated using KL divergence. The calculation formula is as follows:

[0058]

[0059] In the formula, Represented as divergence, This is represented as the current model output probability distribution. This represents the probability distribution output by the historical model. When the divergence value exceeds a preset range, it is determined to be a predicted drift sample.

[0060] The monitoring module calculates a unified sample value score by weighting multiple evaluation results. The unified threshold for the value score is set at 0.50. Samples with scores exceeding the threshold are automatically placed into the labeling cache for temporary storage. Low-value samples are directly discarded, and only a very low percentage (5%) are randomly retained for global data distribution monitoring.

[0061] In step S3, the human-computer interaction annotation module reads the cached images in the annotation buffer area in real time, and provides functions such as rectangular box annotation, single category selection, and secondary verification of annotation results. The annotator accurately selects the defect area and matches the corresponding defect category label. After the annotation is completed and verified to be correct, the structured sample and annotation file are automatically archived to the annotated dataset.

[0062] Step S4: Configure dual incremental training trigger conditions. The first is quantity trigger: the cumulative number of newly labeled samples reaches 200; the second is time trigger: the natural day cycle reaches 7 days. If either condition is met, the system will automatically start the incremental training task.

[0063] Step S5: Load the weights of the online defect detection model and simultaneously enable the anti-forgetting constraint mechanism; use knowledge distillation to help retain historical model knowledge, and the distillation loss calculation formula is as follows:

[0064]

[0065] In the formula, This is expressed as distillation loss. This is represented as the probability distribution of the teacher model (old model). This is represented as the probability distribution of the student model (model to be updated).

[0066] Simultaneously, a flexible weight solidification strategy is employed to apply regularization constraints to key weights of historical tasks; a fixed sampling ratio is used to construct the incremental training set by mixing newly labeled samples with historical replay samples at a fixed ratio of 7:3; incremental fine-tuning adopts a strategy of freezing the backbone feature extraction layer and iterating only on the training parameters of the detection head and shallow layers, with the learning rate fixed at [value missing]. The number of iteration rounds is fixed at 30 rounds, and a lightweight update model is generated after training is completed.

[0067] Step S6: Call the independent standard test set to conduct quantitative verification and fix the core qualification indicators: overall mAP not less than 0.92, false negative rate ≤1.5%, false positive rate ≤1.0%; the model can be judged to be qualified only if all three indicators meet the standards; the qualified model is automatically distributed to replace the old model in the online detection module; after the model is deployed, the unlabeled cache area is automatically cleared to complete the single iteration closed loop.

[0068] Step S7 involves continuously executing the above processes of online detection, high-value sample screening, manual annotation, incremental training, model validation, and deployment to form a long-term closed-loop continuous learning mechanism, enabling the model to dynamically adapt to changes in defect morphology caused by changes in production line processes, raw materials, and equipment operating conditions.

[0069] Corresponding to the aforementioned continuous learning methods in the field of defect detection, this embodiment also provides a continuous learning system in the field of defect detection. The system consists of an online detection module, a monitor module, a sampling and storage module, a human-computer interaction annotation module, an incremental training module, a model verification and deployment module, and a data balancing and replay module.

[0070] The online detection module is deployed at the edge of the industrial production line, integrating an industrial image acquisition unit, an image preprocessing and computing unit, and a model inference and execution unit. It continuously acquires product surface images in real time, performs unified preprocessing, and then sends them to the current defect detection model for inference. It outputs complete defect categories, defect coordinate frames, and confidence parameters. It also performs local data caching and cross-module data push, and transmits images and inference results to the monitoring module and the sampling and storage module, respectively, providing raw data support for subsequent sample evaluation and screening.

[0071] The monitoring module maintains real-time data connection with the online detection module, receiving each frame of detection image and inference results in real time. It has five fixed algorithm logics built in, including uncertainty calculation, feature distance solution, enhanced consistency verification, autoencoder reconstruction operation, and KL divergence drift calculation. It quantifies and scores each sample and labels high-value samples, and sends the labeled sample instructions to the sampling and storage module to achieve accurate screening and diversion of high-value samples.

[0072] The sampling and storage module is responsible for the classification and management of all samples. It receives the screening tags from the monitoring module and stores the samples judged to be high-value separately in a dedicated unlabeled cache area. The cache area adopts a fixed capacity limit, with an upper limit of 500 samples. When it is full, the sample with the lowest value score is automatically removed. The cache queue is maintained by a dual strategy of first-in-first-out and value score priority. Low-value regular samples are discarded in real time, and only a small number of random samples are retained for background data statistics.

[0073] The human-computer interaction annotation module reads images to be annotated from the annotation cache as needed, provides a visual operation interface, and supports functions such as defect selection, category selection, annotation content verification, and abnormal sample remarks. All qualified annotation data that has completed verification is automatically archived and stored in the labeled dataset for structured and unified management, providing compliant annotation samples for incremental training.

[0074] The incremental training module is bound to the labeled dataset and the data balancing and replay module. After the quantity or time trigger conditions are met, the training task is automatically started, the newly labeled samples are automatically retrieved, and the data balancing and replay module is requested to send historical typical defects and difficult examples. The training set is constructed by mixing them in a fixed ratio. The incremental fine-tuning is completed by relying on the dual anti-forgetting strategies of elastic weight solidification and knowledge distillation, and the updated model file with converged parameters is output.

[0075] The model validation and deployment module connects to the incremental training module, automatically loads new model weights and imports them into the standard test set, completes inference testing in batches, automatically calculates key indicators such as mAP, false negative rate, and false positive rate, and makes judgments based on preset pass thresholds. If the validation is successful, the module automatically completes model version replacement, permission verification, and online deployment. If the validation fails, the current training model is automatically discarded and awaits the next round of new sample accumulation and iteration.

[0076] The data balancing and replay module stores typical samples of various defects and historical difficult samples in the long term. In each round of incremental training, representative historical data is extracted in a fixed proportion to participate in mixed training, balancing the learning weights of new and old tasks, suppressing the catastrophic forgetting problem in the incremental update process of the model, and ensuring that the model has a stable ability to identify both early classic defects and newly added variant defects.

[0077] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A continuous learning method in the field of defect detection, characterized in that, Includes the following steps: Step S1: Train an initial defect detection model using the initial labeled dataset, and deploy the initial defect detection model to the online detection module; In step S2, the online detection module performs real-time defect detection on the product images of the production line, outputs defect category, location, confidence information and records the detection results; the monitoring module evaluates the value of the detected samples and calculates the value score, marks the samples with the value score exceeding the preset threshold as high-value samples and sends them to the labeling buffer area. Step S3: Use the human-computer interaction annotation module to select the defect location and annotate the defect category of high-value samples in the annotation cache area. After the annotation is completed, move the samples into the annotated dataset. Step S4: When the number of samples in the labeled dataset reaches a preset threshold or a preset time interval, the incremental training of the defect detection model is triggered. Step S5: Using the anti-forgetting technique of EWC solidification with elastic weights or knowledge distillation, the current defect detection model is incrementally fine-tuned by combining newly labeled data with historical core data to obtain the updated defect detection model. Step S6: Verify the accuracy, false negative rate, and false positive rate of the updated defect detection model. After verification, deploy it to the online detection module to replace the old model, and clear or retain the samples in the unlabeled cache area according to the strategy. Step S7: Repeat steps S2 to S6 to achieve continuous learning of the defect detection model.

2. The method according to claim 1, characterized in that: In step S2, the value assessment strategy includes at least one of uncertainty estimation, boundary sample detection, consistency check, autoencoder reconstruction error, and prediction drift detection.

3. The method according to claim 2, characterized in that: The uncertainty estimation is based on the softmax probability entropy or maximum class probability output by the model. When the confidence level is lower than a preset threshold, it is judged as a high-value sample.

4. The method according to claim 2, characterized in that: The boundary sample detection method involves obtaining the sample embedding vector through a feature extractor, calculating the distance to the nearest neighbor of the existing training set features, and determining the sample as a high-value sample when the distance exceeds a preset threshold.

5. The method according to claim 2, characterized in that: The consistency check involves performing rotation and cropping data augmentation on the same image and then inputting it into the model. If the output results are inconsistent, the image is identified as a high-value sample. The autoencoder reconstruction error involves reconstructing a normal sample using an autoencoder. When the reconstruction error of the input image is greater than a preset threshold, the image is identified as a high-value sample. The prediction drift detection involves comparing the output differences between the current model and historical versions of the model. If the differences are significant, the image is identified as a high-value sample.

6. A continuous learning system in the field of defect detection according to claim 1, characterized in that: Step S5 also includes a data balancing and replay step, which samples typical defects and difficult samples from historical data and mixes them with newly labeled data for training to prevent catastrophic forgetting of the model.

7. A continuous learning system in the field of defect detection, characterized in that, include: An online detection module is used for real-time defect detection and recording of results; A monitoring module, which is used for value assessment and screening of high-value samples; A sampling and storage module, which is used to store high-value samples into a cache area to be labeled; A human-computer interaction annotation module, which is used for defect annotation; An incremental training module, which is used to fine-tune the model using anti-forgetting techniques; The model validation and deployment module is used to validate and deploy the updated model.

8. The system according to claim 7, characterized in that: The monitor module is configured to perform at least one of the following value assessment strategies: uncertainty estimation, boundary sample detection, consistency check, autoencoder reconstruction error, and prediction drift detection.

9. The system according to claim 7, characterized in that: It also includes a data balancing and replay module, which is used to sample representative samples from historical data and mix them with newly labeled data for incremental training to prevent the model from forgetting old, flawed knowledge.

10. The system according to claim 7, characterized in that: The unlabeled cache area is updated with capacity using a first-in-first-out (FIFO) or value score priority strategy; after model deployment, the unlabeled cache area is cleared or some samples are retained according to the strategy.

Images