Defect detection method fusing online incremental learning and semi-supervised graph clustering
By employing online incremental learning and semi-supervised graph clustering methods, automatic annotation and adaptive adjustment are achieved, solving the problems of high annotation costs and process drift in high-value production, and realizing efficient and stable defect detection.
Patent Information
- Application Number
- CN202510816707.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-18
- Publication Date
- 2025-10-31
AI Technical Summary
Existing defect detection systems are costly to label in high-value, low-defect-rate production and are susceptible to process drift, leading to decreased detection accuracy, increased false detection rate, and inability to adapt to changes in flexible manufacturing scenarios.
We employ online incremental learning and semi-supervised graph clustering methods. We construct a highly reliable pseudo-label propagation channel through an automatic labeling mechanism for spectral clustering. Combined with entropy confidence learning and sample selection mechanisms, we achieve self-evaluation and adaptive adjustment. We also introduce feature comparison and replay mechanisms to monitor process drift and update the model.
It enables automatic identification of highly reliable defect samples without manual annotation, improves the robustness of pseudo-labels, reduces computational resource requirements, adapts to dynamic environmental changes, and maintains detection accuracy and stability.
Smart Images
Figure CN120876362A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of product defect detection technology, specifically relating to a defect detection method and system that integrates online incremental learning and semi-supervised graph clustering. Background Technology
[0002] In modern intelligent manufacturing systems, product quality inspection, as a key link in ensuring stable production line operation and yield control, is increasingly becoming an important indicator for measuring the flexibility and autonomy of manufacturing systems, particularly in highly flexible manufacturing scenarios such as customized electronics manufacturing, complex precision machining, and high-frequency iteration consumer electronics production lines. Traditional defect detection systems based on fixed models have revealed serious limitations in adaptability.
[0003] Most current mainstream defect detection methods rely on supervised learning frameworks, typically requiring a large number of manually labeled defect samples for training to build deep neural network models for feature extraction and classification of defect images. However, in actual industrial settings, two significant obstacles exist: First, the cost of labeling defect samples is extremely high, especially in high-value, low-defect-rate production, where only a very small number of defect samples are found among thousands of images. Second, the uncertainty of production processes and the variability of raw material batches often cause a significant deviation between the feature distribution learned during the detection model training and the actual operating conditions, a phenomenon known as process drift. This drift can manifest as subtle changes in the surface texture of raw materials, inconsistencies in lighting conditions, variations in camera parameters, or differences in product appearance caused by changes in process control temperature. In systems without continuous feedback and model adaptation mechanisms, this type of drift usually directly leads to decreased detection accuracy, increased false positive rates, and even large-scale missed detections. Summary of the Invention
[0004] In view of the above-mentioned problems, this invention provides a defect detection method that integrates online incremental learning and semi-supervised graph clustering. It adopts an online automatic sample labeling mechanism based on spectral clustering to establish a highly reliable pseudo-label propagation channel, breaking through the bottleneck of traditional manual labeling and thus enabling real-time operation. At the same time, it adopts an entropy-based confidence learning and sample screening mechanism to realize the self-evaluation capability of classification output, introduces an information theory interpretation path for deep models, and greatly improves the robustness of pseudo-labels.
[0005] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:
[0006] A defect detection method integrating online incremental learning and semi-supervised graph clustering includes the following steps:
[0007] S10, acquires continuous production line image stream through online acquisition module. Where, xi For the i-th sample image;
[0008] S20 uses the model to make a preliminary judgment on the sample images in S10, filters out samples that are judged to be abnormal but with low confidence, and constructs a candidate sample set.
[0009] S30 constructs a graph structure based on the samples from S20 and automatically labels the samples based on spectral clustering;
[0010] S40, Construct a confidence evaluation function, and define the confidence of each prediction result based on classification entropy to screen reliable samples;
[0011] S50 uses a confidence-weighted cross-entropy loss function to incrementally update the model during training.
[0012] In step S30, feature representations of each image in the candidate set are extracted. Construct a feature similarity graph G = (V, E), where V = {f(x)} i Let E be the set of edges, and let the edge weights be defined as cosine similarity.
[0013]
[0014] Construct a Laplacian matrix L = DW, where D is the degree matrix and W is the weight matrix. Then, perform spectral clustering to divide the samples into K latent classes C1, C2, ..., Cn. K Select the sample with the highest centrality in each category. As a pseudo-tag anchor, it is defined as:
[0015]
[0016] After selecting the anchor point, set its corresponding pseudo tag. Assign all samples in this category and add them to the pseudo-label set.
[0017] In one possible implementation, S40, a confidence evaluation function is constructed, defining the confidence level of each prediction result based on classification entropy to filter reliable samples:
[0018]
[0019] Wherein, H(x) i ) is the sample x i The predicted entropy, p ic For the model on sample x i Predicted probability of belonging to category c; introducing a confidence threshold θ H Define a set of trusted pseudo-tags:
[0020]
[0021] In one possible implementation, in S50, the confidence-weighted cross-entropy loss function during incremental training... The definition is as follows:
[0022]
[0023] in, The sample weighting factor is based on entropy; the model update employs a fine-tuning strategy.
[0024] Among them, W t W t+1 These are the model parameters at iterations t and t+1. The gradient of the loss function with respect to the parameter W;
[0025] η is the learning rate, used to control the step size for parameter updates.
[0026] One possible implementation also includes:
[0027] S60, online process drift monitoring, to trigger model adjustments for adaptive process drift;
[0028] The Fréchet distance is used to measure the change in the feature distribution between the old and new samples; let the feature distribution of the samples in the past time period T1 be... The current sample distribution for time period T2 is as follows: Define the distribution drift distance as:
[0029]
[0030] When D F >∈ drift At that time, the model structure retraining mechanism is triggered, and the pseudo-label strategy and threshold function are updated simultaneously.
[0031] One possible implementation is to introduce feature contrast loss into the loss function, assuming that the high-confidence samples in the current training batch are... Its feature embedding is represented as Then for any positive and negative sample pair The contrastive loss function is defined as follows:
[0032]
[0033] in This represents the normalized cosine similarity, where τ is the temperature coefficient. This refers to the set of all samples in the current mini-batch.
[0034] The main loss function for the training objective is:
[0035]
[0036] Where W0 represents the initial pre-training weights, λ controls the intensity of the model update offset, and β is the weighting coefficient of the contrastive loss.
[0037] In one possible implementation, the loss function also incorporates a heterogeneous memory replay mechanism, which maintains a dynamic sample memory. This includes samples from historical high-confidence sets. Representative samples selected from In each training round, a certain proportion of replay samples are uniformly sampled from the memory bank and used together with the current training samples to participate in the training, forming a joint optimization objective:
[0038]
[0039] Where, γ i The sample weights are initially set to 1 and can be updated using an exponential decay method during long-term iterations.
[0040] The loss function for the overall training objective is:
[0041]
[0042] in, For the complete final loss function, The loss function is trained using historical samples from the memory bank. The main loss function, where η1 is the weighting coefficient of the memory playback loss term.
[0043] The present invention has at least the following beneficial effects:
[0044] 1. This invention constructs a closed-loop link of "perception-evaluation-learning-monitoring-relearning", covering the entire life cycle learning process from image acquisition to model self-evolution. It is the first to synergistically apply multi-source information entropy, graph clustering and feature drift index to the engineering integration of visual detection tasks, and has complete patent technology barriers and high engineering transformation value.
[0045] 2. An online automatic sample labeling mechanism based on spectral clustering is introduced, which for the first time introduces a centrality selection strategy on graph structure in industrial testing, establishes a highly reliable pseudo-label propagation channel, breaks through the bottleneck of traditional manual labeling, and has real-time operation capability.
[0046] 3. An entropy-based confidence learning and sample selection mechanism is introduced to enable the self-evaluation capability of classification output, introduce an information theory interpretation path for deep models, and greatly improve the robustness of pseudo-labels.
[0047] 4. Introduce an incremental learning process with feature comparison and replay mechanisms to construct a joint loss:
[0048]
[0049] By strengthening the model's discrimination boundaries through contrastive learning and enhancing model stability through a memory bank, a lightweight learning path that does not rely on retraining with large samples is formed.
[0050] 5. A process drift monitoring mechanism based on Fréchet distance was established, and dynamic monitoring indicators were constructed:
[0051]
[0052] It can accurately capture changes in the production line environment and trigger adaptive strategies through dynamic drift sensing to adapt to the dynamic industrial environment. Attached Figure Description
[0053] Figure 1 This is a flowchart illustrating the steps of a defect detection method that integrates online incremental learning and semi-supervised graph clustering, according to an embodiment of the present invention. Detailed Implementation
[0054] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0055] In recent years, although a few studies have begun to explore the application of semi-supervised and self-supervised learning in industrial vision, such as using unlabeled data to assist training or introducing pseudo-labels to optimize models, these solutions have not yet solved the core problem: how to build a detection system that can learn sustainably and evolve autonomously, so that it can continuously optimize and evolve through real-time feedback in real production line operation, thereby adapting to the dynamic data distribution and defect patterns in the long term.
[0056] Therefore, there is an urgent need for a detection system with the following capabilities: (1) It can automatically identify and screen out high-confidence defect samples from the production line without manual labeling; (2) It can quantitatively evaluate the output confidence of the current model to avoid overfitting caused by including low-quality pseudo-labels in the training; (3) It can incrementally learn and update the existing model after new samples arrive, instead of completely retraining, thereby saving computing resources; (4) It can sense changes in data distribution, promptly determine whether process drift has occurred, and make model adjustment strategies.
[0057] To solve the above technical problems, see Figure 1The image shows a defect detection method that integrates online incremental learning and semi-supervised graph clustering according to an embodiment of the present invention, comprising the following steps:
[0058] S10, acquires continuous production line image stream through online acquisition module. Where, x i For the i-th sample image;
[0059] S20 uses the model to make a preliminary judgment on the sample images in S10, filters out samples that are judged to be abnormal but with low confidence, and constructs a candidate sample set.
[0060] S30 constructs a graph structure based on the samples from S20 and automatically labels the samples based on spectral clustering;
[0061] S40, Construct a confidence evaluation function, and define the confidence of each prediction result based on classification entropy to screen reliable samples;
[0062] S50 uses a confidence-weighted cross-entropy loss function to incrementally update the model during training.
[0063] In step S30, feature representations of each image in the candidate set are extracted. Construct a feature similarity graph G = (V, E), where V = {f(x)} i Let E be the set of edges, and let the edge weights be defined as cosine similarity.
[0064]
[0065] Construct a Laplacian matrix L = DW, where D is the degree matrix and W is the weight matrix. Then, perform spectral clustering to divide the samples into K latent classes C1, C2, ..., Cn. K Select the sample with the highest centrality in each category. As a pseudo-tag anchor, it is defined as:
[0066]
[0067] After selecting the anchor point, set its corresponding pseudo tag. Assign all samples in this category and add them to the pseudo-label set.
[0068] The above technical solution introduces an online automatic sample labeling mechanism based on spectral clustering, and for the first time introduces a centrality selection strategy on a graph structure in industrial inspection, defining the anchor point formula:
[0069]
[0070] This establishes a highly reliable pseudo-label propagation channel, breaking through the bottleneck of traditional manual labeling and enabling real-time operation.
[0071] Furthermore, to control the quality of pseudo-labels and prevent the model from overfitting to incorrect pseudo-labels during training, S40, a confidence evaluation function is constructed, defining the confidence of each prediction result based on classification entropy to filter reliable samples:
[0072]
[0073] Wherein, H(x) i ) is the sample x i The predicted entropy, p ic For the model on sample x i Predicted probability of belonging to category c; introducing a confidence threshold θ H Define a set of trusted pseudo-tags:
[0074]
[0075] In the above technical solution, the confidence learning and sample selection mechanism based on entropy enables the self-evaluation capability of the classification output, introduces an information theory interpretation path for deep models, and greatly improves the robustness of pseudo-labels.
[0076] Furthermore, in S50, the confidence-weighted cross-entropy loss function in incremental training... The definition is as follows:
[0077]
[0078] in, The sample weighting factor is based on entropy; the model update employs a fine-tuning strategy.
[0079]
[0080] Among them, W t W t+1 These are the model parameters at iterations t and t+1. The gradient of the loss function with respect to the parameter W;
[0081] η is the learning rate, used to control the step size for parameter updates.
[0082] Furthermore, to ensure that the system's performance does not degrade due to changes in new data distribution during long-term operation, it also includes:
[0083] S60, online process drift monitoring, to trigger model adjustments for adaptive process drift;
[0084] The Fréchet distance is used to measure the change in the feature distribution between the old and new samples; let the feature distribution of the samples in the past time period T1 be... The current sample distribution for time period T2 is as follows: Define the distribution drift distance as:
[0085]
[0086] When D F >∈ drift At that time, the model structure retraining mechanism is triggered, and the pseudo-label strategy and threshold function are updated simultaneously.
[0087] In the above technical solution, a process drift monitoring mechanism based on Fréchet distance is used to construct dynamic monitoring indicators. This allows for precise capture of changes in the production line environment, triggering adaptive strategies through dynamic drift sensing to adapt to real-world dynamic industrial environments.
[0088] Furthermore, to enhance the controllability and robustness of the self-learning detection system's online learning, especially under conditions of low sample density or process drift, and to maintain the model's generalization ability, a feature contrast loss is introduced into the loss function. Let the high-confidence samples in the current training batch be... Its feature embedding is represented as Then for any positive and negative sample pair The contrastive loss function is defined as follows:
[0089]
[0090] in This represents the normalized cosine similarity, where τ is the temperature coefficient. This refers to the set of all samples in the current mini-batch.
[0091] The main loss function for the training objective is:
[0092]
[0093] Where W0 represents the initial pre-training weights, λ controls the intensity of the model update offset, and β is the weighting coefficient of the contrastive loss.
[0094] Furthermore, to prevent the model from completely forgetting the feature distribution of old samples when new data arrives, a heterogeneous memory replay mechanism is introduced into the loss function. This mechanism maintains a dynamic sample memory. This includes samples from historical high-confidence sets. Representative samples selected from In each training round, a certain proportion of replay samples are uniformly sampled from the memory bank and used together with the current training samples to participate in the training, forming a joint optimization objective:
[0095]
[0096] Where, γ i The sample weights are initially set to 1 and can be updated using an exponential decay method during long-term iterations.
[0097] The loss function for the overall training objective is:
[0098]
[0099] in, For the complete final loss function, The loss function is trained using historical samples from the memory bank. The main loss function, where η1 is the weighting coefficient of the memory playback loss term.
[0100] The above technical solution introduces an incremental learning process with feature comparison and replay mechanisms, and constructs a joint loss:
[0101]
[0102] By strengthening the model's discrimination boundaries through contrastive learning and enhancing model stability through a memory bank, a lightweight learning path that does not rely on retraining with large samples is formed.
[0103] Furthermore, during training, the model structure supports freezing some low-level perceptual modules and updating only the high-level feature representation parts, thereby improving computational efficiency and structural stability. Let W = [W low W high ],but:
[0104]
[0105] This hierarchical training strategy effectively reduces the oscillations and degradation caused by "full parameter learning" when adapting the model to new scenarios.
[0106] The above technical solution introduces a parameter-freezing structure update strategy and defines parameter update constraints:
[0107]
[0108] This enables hierarchical training of high and low layer parameters of the model, improves system operating efficiency and update controllability, and adapts to edge computing scenarios.
[0109] In addition, in the online process drift monitoring section, the system periodically calculates the characteristic mean μ. t With covariance Σ t Compare the changing trends of the characteristic distribution within a time window T. When any two adjacent time periods T... i ,T i+1 The Fréchet distance satisfies:
[0110] D F (T i ,T i+1 )>∈ drift
[0111] The system then enters "drift response mode" and automatically triggers the following operation chain:
[0112] Recalculate the graph clustering structure of the current candidate sample set;
[0113] Reinitialize the current learning rate and pseudo-label confidence threshold θ H ;
[0114] Load the initial model parameters W0 and retain the current training parameters as the comparison record W. hist ;
[0115] Rebuild memory bank The sampling strategy was updated to sort by the dimension of the largest sample change in the drift direction.
[0116] The mechanism described above in this invention combines a closed-loop strategy of "drift monitoring – fine-tuning and resetting – historical comparison" to significantly improve the stability of the system during long-term operation.
[0117] Throughout the system deployment, all of the above mechanisms can be independently deployed and encapsulated as microservices through modular APIs, supporting both edge computing nodes and centralized cloud training architectures. Real-time processing components, such as H(x)... i ), sample features f(x) i Retraining and memory playback can be done on the local GPU, while retraining and clustering modules can be performed asynchronously on the server.
[0118] The parameters mentioned in this article are explained below:
[0119] Symbols related to samples and features:
[0120] x i : The i-th sample image.
[0121] The collection of images acquired online, i.e., the input image stream.
[0122] The set of candidate suspected defect samples is generated by the model's initial judgment.
[0123] A set of highly reliable pseudo-labeled samples after confidence level filtering.
[0124] The pseudo-labeled sample set is labeled based on the clustering results.
[0125] A set of high-confidence samples from the historical training process.
[0126] The sample memory stores representative samples used for replaying training.
[0127] C k : The kth cluster (graph clustering results).
[0128] The anchor sample in the kth cluster (with the strongest centrality).
[0129] Sample x i The pseudo-labels are assigned by the clustering results.
[0130] Feature representation and similarity calculation:
[0131] Image sample x i The embedded feature vector (with dimension d).
[0132] w ij Sample x i With x j The feature similarity (edge weight) between them is calculated using cosine similarity.
[0133] sim(a,b): The normalized cosine similarity of vectors a and b, i.e.:
[0134]
[0135] Graph structure and clustering:
[0136] G = (V, E): Graph structure, where V is the set of nodes (sample features) and E is the set of edges.
[0137] W: The weight matrix of the graph, with elements w ij .
[0138] D: Degree matrix, with diagonal elements representing node degrees.
[0139] L = DW: Graph Laplacian matrix, used for spectral clustering.
[0140] Confidence assessment and pseudo-label screening:
[0141] H(x i ): Sample x i The prediction entropy (uncertainty assessment) is calculated using the following formula:
[0142]
[0143] p ic Sample x iThe probability of being predicted as class c comes from the softmax output.
[0144] C: Total number of categories.
[0145] θ H Confidence threshold, used to filter out reliable pseudo-labels.
[0146] Sample weights are adaptively calculated based on the predicted entropy (the smaller the entropy, the larger the weight). Loss function related:
[0147] Confidence-weighted cross-entropy loss function in incremental training.
[0148] The contrast loss function is used to strengthen the boundaries between classes.
[0149] τ: Temperature scaling factor in contrast loss, used to adjust the similarity distribution.
[0150] The set of all samples in the current training batch.
[0151] Training loss from historical samples in the memory bank.
[0152] γ i The weights of the samples in the memory bank can be determined using an exponential decay strategy.
[0153] Main loss function, combined increment, contrast, and regularization term:
[0154]
[0155] β: Weighting coefficient for contrast loss.
[0156] λ: Coefficient of the regularization term in the model parameters.
[0157] W: The current set of model parameters.
[0158] W0: Initial model parameters (pre-trained).
[0159] The complete final loss after adding memory bank replay:
[0160]
[0161] η1: Weighting coefficient of the memory playback loss term.
[0162] Process drift detection:
[0163] μ1, μ2: The characteristic mean of the samples within time periods T1 and T2.
[0164] Σ1, Σ2: Feature covariance matrices of samples within time periods T1 and T2.
[0165] Fréchet distance between two characteristic distributions:
[0166] D F =‖μ1-μ2‖ 2 +Tr(Σ1+Σ2-2(Σ1Σ2) 1 / 2 )
[0167] ∈ drift : Drift response threshold, when D F >∈ drift Trigger the model reset strategy.
[0168] Parameter update and training strategies:
[0169] W low W high : These represent the sets of parameters for the lower and higher levels of the model, respectively.
[0170] This indicates that the underlying perception parameters are frozen, and only the higher-level parameters are updated.
[0171] W t W t+1 : Model parameters at iterations t and t+1.
[0172] The gradient of the loss function with respect to the parameter w.
[0173] η: Learning rate, used to control the step size for parameter updates.
[0174] It should be understood that the exemplary embodiments described herein are illustrative and not restrictive. Although one or more embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art will understand that various changes in form and detail may be made without departing from the spirit and scope of the invention as defined by the appended claims.
Claims
1. A defect detection method integrating online incremental learning and semi-supervised graph clustering, characterized in that, Includes the following steps: S10, acquires continuous production line image stream through online acquisition module. Where, x i For the i-th sample image; S20 uses the model to make a preliminary judgment on the sample images in S10, filters out samples that are judged to be abnormal but with low confidence, and constructs a candidate sample set. S30 constructs a graph structure based on the samples from S20 and automatically labels the samples based on spectral clustering; S40, Construct a confidence evaluation function, and define the confidence of each prediction result based on classification entropy to screen reliable samples; S50 uses a confidence-weighted cross-entropy loss function to incrementally update the model during training. In step S30, feature representations of each image in the candidate set are extracted. Construct a feature similarity graph G = (V, E), where V = {f(x)} i Let E be the set of edges, and let the edge weights be defined as cosine similarity. Construct a Laplacian matrix L = DW, where D is the degree matrix and W is the weight matrix. Then, perform spectral clustering to divide the samples into K latent classes C1, C2, ..., Cn. K Select the sample with the highest centrality in each category. As a pseudo-tag anchor, it is defined as: After selecting the anchor point, set its corresponding pseudo tag. Assign all samples in this category and add them to the pseudo-label set.
2. The defect detection method integrating online incremental learning and semi-supervised graph clustering as described in claim 1, characterized in that, S40, Construct a confidence evaluation function, defining the confidence level of each prediction result based on classification entropy to filter reliable samples: Wherein, H(x) i ) is the sample x i The predicted entropy, p ic For the model on sample x i Predicted probability of belonging to category c; introducing a confidence threshold θ H Define a set of trusted pseudo-tags:
3. The defect detection method integrating online incremental learning and semi-supervised graph clustering as described in claim 2, characterized in that, In S50, the confidence-weighted cross-entropy loss function in incremental training The definition is as follows: in, The sample weighting factor is based on entropy; the model update employs a fine-tuning strategy. Among them, W t W t+1 These are the model parameters at iterations t and t+1. Let W be the gradient of the loss function with respect to the parameter W. η is the learning rate, used to control the step size for parameter updates.
4. The defect detection method integrating online incremental learning and semi-supervised graph clustering as described in claim 3, characterized in that, Also includes: S60, online process drift monitoring, to trigger model adjustments for adaptive process drift; The Fréchet distance is used to measure the change in the feature distribution between the old and new samples; let the feature distribution of the samples in the past time period T1 be... The current sample distribution for time period T2 is as follows: Define the distribution drift distance as: When D F >∈ drift At that time, the model structure retraining mechanism is triggered, and the pseudo-label strategy and threshold function are updated simultaneously.
5. The defect detection method integrating online incremental learning and semi-supervised graph clustering as described in claim 4, characterized in that, The loss function introduces feature contrast loss, assuming that the high-confidence samples in the current training batch are... Its feature embedding is represented as Then for any positive and negative sample pair The contrastive loss function is defined as follows: in This represents the normalized cosine similarity, where τ is the temperature coefficient. This refers to the set of all samples in the current mini-batch. The main loss function for the training objective is: Where W0 represents the initial pre-training weights, λ controls the intensity of the model update offset, and β is the weighting coefficient of the contrastive loss.
6. The defect detection method integrating online incremental learning and semi-supervised graph clustering as described in claim 5, characterized in that, The loss function also introduces a heterogeneous memory replay mechanism, which maintains a dynamic sample memory. This includes samples from historical high-confidence sets. Representative samples selected from In each training round, a certain proportion of replay samples are uniformly sampled from the memory bank and used together with the current training samples to participate in the training, forming a joint optimization objective: Where, γ i The sample weights are initially set to 1 and can be updated using an exponential decay method during long-term iterations. The loss function for the overall training objective is: in, For the complete final loss function, The loss function is trained using historical samples from the memory bank. The main loss function, where η1 is the weighting coefficient of the memory playback loss term.