A method and system for incremental defect detection based on selective distillation

Abstract

The application discloses a kind of based on selective distillation's incremental defect detection method and system, the method includes the following steps: constructing teacher model and student model, the teacher model and student model are based on the same defect detection network structure;Old defect category sample and label are input into teacher model for training, and the network parameter of well-trained is saved as the initialization weight of student model training;During the student model training process, different classification distillation, regression distillation and feature distillation strategies are executed to the output of teacher model and student model;Student model is trained under the condition of no old defect sample by three kinds of distillation strategies;The defect sample to be detected is input into the student model;The defect category and its position coordinates of the defect sample are output by student model.The application can guarantee old category detection performance while realizing rapid learning and high-precision detection of new defect category.

Description

Technical Field

[0001] This invention belongs to the field of machine vision inspection and artificial intelligence technology, and specifically relates to an incremental defect detection method and system based on selective distillation. Background Technology

[0002] In modern manufacturing, surface defects in industrial products not only affect their appearance but can also lead to decreased mechanical performance and even potential safety hazards. With the continuous development of manufacturing processes and material systems, new types of surface defects are constantly emerging. How to efficiently and accurately detect and identify these defects has become a crucial link in ensuring product quality and production safety. Traditional surface defect detection methods mainly include manual inspection and detection methods based on traditional image processing. Manual inspection relies on the operator's experience and visual judgment, resulting in low efficiency, high subjectivity, and poor stability, making it difficult to meet the needs of modern large-scale production. Detection methods based on traditional image processing typically achieve defect identification through algorithms such as edge detection, texture analysis, and threshold segmentation. However, these methods are extremely sensitive to lighting, noise, and complex backgrounds, and feature design relies on expert experience, lacking adaptability and robustness to complex and diverse defects.

[0003] In recent years, deep learning-based defect detection methods have gradually emerged. These methods, relying on the powerful feature extraction and representation capabilities of convolutional neural networks, can achieve end-to-end defect localization and classification with the support of large-scale labeled data, demonstrating significantly better detection accuracy and robustness than traditional methods. Currently, mainstream deep learning detectors include one-stage detectors (such as the YOLO series, RetinaNet, GFL, etc.) and two-stage detectors (such as Faster R-CNN, etc.), which have been widely used in industrial defect detection tasks.

[0004] Chinese invention patent application CN120431069A proposes a method for detecting multiple types of surface defects in industrial products based on an improved YOLOv5 model. This method improves the model's detection accuracy, convergence speed, and cross-scene transfer capability by introducing generative adversarial networks to augment the dataset, embedding an attention mechanism in the YOLOv5 backbone network, employing a decoupled detection head, and using an improved transfer learning algorithm. This allows the method to better address the challenges of defect diversity and complexity.

[0005] The aforementioned solutions still focus on data augmentation and transfer learning, and they have limitations in addressing the incremental learning problem of new defect categories: their improved solutions rely on a complete data augmentation process, failing to solve the problem of decreased detection accuracy of old categories when new defect categories are added; in incremental learning scenarios, it is difficult to balance the learning of new categories with the preservation of old categories; in complex industrial environments, issues such as blurred defect boundaries and imbalanced sample sizes may still affect the model's accuracy and stability in locating defect targets. Therefore, how to achieve rapid learning and high-precision detection of new defect categories while ensuring the detection performance of old categories remains a problem that existing technologies urgently need to solve. Summary of the Invention

[0006] This invention provides an incremental defect detection method and system based on selective distillation, aiming to solve the problems of existing technologies such as a significant decrease in the detection accuracy of old categories when adding new defect categories, difficulty in balancing the detection performance of new and old categories in incremental learning scenarios, and insufficient positioning accuracy and detection stability caused by blurred defect boundaries and sample imbalance in complex industrial environments.

[0007] To address the aforementioned technical problems, this invention proposes an incremental defect detection method based on selective distillation, comprising the following steps: Construct a teacher model and a student model, which are based on the same defect detection network structure; The old defect category samples and labels are input into the teacher model for training, and the trained network parameters are saved as the initial weights for training the student model. During the training of the student model, the following distillation strategies are performed on the outputs of the teacher model and the student model: On the classification head, classification distillation with mean squared error loss is performed based on high-confidence anchor regions; on the regression head, regression distillation with L2 loss is performed based on high-quality anchor selection in a two-stage cascade; on the feature pyramid network, feature distillation with KL divergence loss is performed based on an IoU-driven difficulty-aware mechanism. The student model was trained using three distillation strategies under conditions without old defective samples. Input the defect sample to be detected into the student model; the student model outputs the defect category and its location coordinates of the defect sample.

[0008] Preferably, both the teacher model and the student model use the same defect detection network GFLv1, and both use ResNet50 as the backbone network and FPN as the neck network.

[0009] Preferably, the training method for the student model is as follows: S1: Input the new defect category samples into the student model to obtain the prediction results of the classification head, regression head, and FPN; S2: Feed the same input samples into the teacher model at the same time to obtain the corresponding classification head, regression head and FPN outputs; S3: Align the outputs of the teacher model and the student model on the classification head, regression head and FPN respectively, and design corresponding distillation strategies for each, and calculate the distillation loss of the three distillation strategies. S4: The student model's own loss and distillation loss are weighted and combined to form the total loss, and the student model parameters are updated through backpropagation; S5: Repeat steps S1–S4 until training converges to obtain a student model capable of simultaneously recognizing both old and new defect categories.

[0010] Preferably, the fractional distillation method specifically comprises: Output of the teacher model classification head Softmax normalization is performed on the category channel dimension, and the maximum value is taken as the confidence value C for each anchor point; Calculate the mean on the confidence level C distribution. and variance And construct an adaptive threshold based on the following formula. :

[0011] in The proportionality coefficient is a hyperparameter, 0 < <1; Select those with confidence levels higher than the adaptive threshold. The set of anchor points is used as the classification distillation region, within which the L2 loss between the teacher model classification head and the student model classification head in the old defect category output is calculated, which serves as the classification distillation loss function.

[0012] Preferably, the loss function for the fractional distillation is:

[0013] In the formula, For the classification distillation loss function, The output of the old defect category in the student model classification head, CR is the classification distillation region.

[0014] Preferably, the regression distillation method specifically includes: For each ground truth bounding box, select the Top-K anchor points that are closest to its center to form a candidate anchor point set; Calculate the mean of the IoU distribution of the candidate anchor point set. and standard deviation And construct an adaptive threshold based on the following formula. :

[0015] in The proportionality coefficient is a hyperparameter, 0 < <1; Select an IoU value higher than the adaptive threshold. The set of anchor points is used as the regression distillation region; Within the regression distillation region, a softmax operation is performed on the output of the teacher model regression head, and a log_softmax operation is performed on the output of the student model regression head. The distillation loss is then calculated based on the IoU-weighted KL divergence.

[0016] Preferably, the loss function of the regression distillation is:

[0017] In the formula, The loss function for regression distillation, For the output of the regression head of the teacher model, The output of the regression head for the student model. This indicates the number of anchor frames in the regressive distillation region. For the return distillation region, This represents the IoU value between the anchor point and the matched ground truth bounding box. Let represent the KL divergence function, and t represent the softening temperature factor.

[0018] Preferably, the characteristic distillation method specifically includes: Calculate the IoU matrix between all anchor boxes and all ground truth boxes, and take the maximum IoU value of the ground truth boxes that match each anchor box to form a vector I, and take the maximum IoU value of all anchor boxes that match each ground truth box to form a vector G; An adaptive threshold vector is constructed based on the following formula. :

[0019] in The proportionality coefficient is a hyperparameter, 0 < <1; ⊗ indicates element-wise multiplication; Select an IoU value higher than the adaptive threshold vector. The set of anchor points is used as the feature distillation region; Within the feature distillation region, the MSE loss is calculated for the outputs of the teacher model FPN and the student model FPN, and used as the feature distillation loss function.

[0020] Preferably, the characteristic distillation loss function is:

[0021] In the formula, The characteristic distillation loss function, The output of the teacher model FPN, The output of the student model FPN, This indicates the number of anchor frames in the characteristic distillation region. This is a characteristic distillation region.

[0022] In another aspect, the present invention also proposes an incremental defect detection system based on selective distillation, the system being used to implement the incremental defect detection method as described in the first aspect of the present invention, comprising: The data input module is used to receive old defect category samples and new defect category samples, and input the samples and their labels into the teacher model and student model; The teacher model module is used to train old defect category samples based on the same defect detection network structure and output the prediction results of the classification head, regression head and feature pyramid network. The student model module, whose initial weights are inherited from the network parameters of the teacher model, is used for training on the detection of new defect category samples; The distillation strategy execution module is used to execute the following distillation strategy based on the outputs of the teacher model and the student model during the student model training process: At the classification head, classification distillation with mean squared error loss is performed based on the high-confidence anchor point region; At the regression head, regression distillation of KL divergence loss is performed based on high-quality anchor point screening in a two-stage cascade. On the feature pyramid network, feature distillation with mean squared error loss is performed based on an IoU-driven difficulty-aware mechanism. The model training module is used to weight and combine the self-detection loss and distillation loss of the student model to form the total loss, and update the parameters of the student model through backpropagation until the training converges. The defect detection module is used to input the defect sample to be detected into the trained student model and output the defect category and its location coordinates of the defect sample.

[0023] Compared with the prior art, the present invention has the following technical effects: 1. Under the same conditions, the incremental defect detection method proposed in this invention greatly improves the detection accuracy of old defect categories compared with the fine-tuning method, effectively mitigating catastrophic forgetting. Compared with other incremental defect detection methods, this method has advantages in the comprehensive detection accuracy of both new and old categories, and is more suitable for industrial production deployment requirements.

[0024] 2. The incremental defect detection method proposed in this invention provides a high-confidence classification distillation strategy, which can perform distillation only in semantically confused regions through confidence screening, avoid invalid alignment of low-confidence background anchors, effectively retain the discriminative knowledge of the old category, and improve the stability of the new category learning process.

[0025] 3. The incremental defect detection method proposed in this invention provides a two-stage cascaded regression distillation strategy. It selects high-quality anchor points through "geometric prior coarse screening + statistical fine screening" and combines IoU-weighted KL divergence distillation to effectively filter noisy anchor points and completely transmit probability distribution information under fuzzy boundaries, thereby improving the localization accuracy and training robustness of the student model.

[0026] 4. The incremental defect detection method proposed in this invention provides an IoU-driven difficulty-aware feature distillation strategy. Based on IoU, the regression difficulty of the defect target is quantified, and distillation resources are adaptively allocated. Priority is given to high-difficulty and critical feature regions, thereby achieving a balance between preserving old knowledge and learning new knowledge, and effectively improving the overall performance of incremental defect detection. Attached Figure Description

[0027] Figure 1 This is a schematic flowchart of the incremental defect detection method described in this invention; Figure 2 This is a schematic diagram of the teacher model and student model described in an embodiment of the present invention; Figure 3 This is a flowchart of the high-confidence classification distillation strategy described in an embodiment of the present invention; Figure 4 This is a flowchart of the two-stage cascaded regression distillation strategy described in the embodiments of the present invention; Figure 5 This is a flowchart of the IoU-driven difficulty-aware feature distillation strategy described in this embodiment of the invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below in conjunction with specific embodiments of this application and with reference to the accompanying drawings. In the description of the specific embodiments, the corresponding terms are explained as follows: GFL, short for Generalized Focal Loss, is a loss function used in machine learning, particularly in deep learning object detection tasks. Its core idea is to introduce adjustable parameters to model the confidence distribution of sample class predictions, building upon traditional cross-entropy and focal loss functions. This achieves a unified representation for classification and localization tasks. By introducing probability distribution constraints into the loss function, GFL establishes a connection between classification and regression tasks, enabling the model to simultaneously learn object class information and bounding box accuracy. Compared to traditional methods, the Generalized Focal Loss function has advantages in handling imbalanced samples and improving detection accuracy.

[0029] ResNet50, short for Residual Network 50, is a deep convolutional neural network model belonging to the ResNet family of networks. This model introduces "residual blocks," adding skip connections across layers to mitigate problems such as vanishing gradients and degradation during deep neural network training. ResNet50 contains 50 layers (including convolutional, pooling, and fully connected layers) and exhibits high feature extraction capabilities and wide applicability in tasks such as image classification, object detection, and semantic segmentation.

[0030] The backbone and neck are two crucial components of a neural network architecture. The backbone is the core of the entire network, primarily responsible for extracting features from the input data. It typically consists of multiple convolutional layers or other feature extraction layers, progressively extracting high-level features from the input data. In image processing tasks, the backbone can extract global and local features, such as edges, textures, and shapes. The neck, located between the backbone and the head, connects and integrates features. It further processes and fuses the features extracted by the backbone, enhancing feature representation and providing higher-quality input for subsequent tasks. The neck typically includes modules such as dimensionality reduction, pooling layers, or attention mechanisms.

[0031] FPN, short for Feature Pyramid Network, is a commonly used deep learning model architecture primarily for tasks such as object detection and image segmentation. Its basic idea is to use a bottom-up convolutional neural network to extract multi-scale features and then fuse high-level semantic information with low-level fine-grained features through top-down paths and lateral connections, thereby obtaining feature representations with strong semantic expressive power at multiple scales simultaneously. FPN can effectively improve the detection performance of targets of different sizes.

[0032] IoU, short for Intersection over Union, is a metric used to measure the degree of overlap between two bounding boxes, commonly used in object detection tasks. It is calculated as the ratio of the intersection area to the union area of ​​the predicted and ground truth bounding boxes, with a value ranging from 0 to 1. A higher IoU value indicates a greater degree of overlap between the predicted and ground truth boxes, resulting in more accurate predictions.

[0033] KL divergence, short for Kullback-Leibler divergence, is a commonly used measure of the difference between probability distributions, used to measure the difference between one probability distribution and another. Its basic definition is:

[0034] in, Indicates the true distribution or the target distribution. KL divergence represents an approximate or model distribution. The closer two distributions are, the smaller the KL divergence value; if they are completely identical, the KL divergence is 0. KL divergence is commonly used in machine learning model training scenarios such as distribution alignment, feature distillation, and probabilistic modeling.

[0035] mAP, short for Mean Average Precision, is a commonly used comprehensive evaluation metric in object detection to measure the detection performance of a model across different categories. It is calculated as follows: first, the mean precision (AP) is calculated for each category, which is the average precision at different recall rates; then, the AP values ​​for all categories are averaged to obtain the mAP value. A higher mAP value indicates better overall model performance in object detection tasks.

[0036] The FSCE method (Foreground–background Separation Contrastive Learning for Object Detection) is a few-shot object detection method based on contrastive learning. Its core idea is to improve the model's detection accuracy for new object categories under limited sample conditions by enhancing the ability to distinguish between foreground and background categories in the feature space.

[0037] The IKLF (Incremental Knowledge Learning Framework) is an incremental learning framework primarily used in defect detection tasks to resolve the conflict between "learning new categories" and "preserving old categories." This method utilizes mechanisms such as knowledge distillation and feature alignment to minimize the forgetting of already learned categories when introducing new ones.

[0038] The ERD (Enhanced Residual Distillation) method is an improved feature distillation method that enhances the ability of student models to inherit features from teacher models by introducing an enhanced distillation strategy into the residual structure, thereby balancing detection accuracy and model generalization ability in defect detection tasks.

[0039] Example 1 This embodiment describes an incremental defect detection method based on selective distillation, such as... Figure 1 As shown, it includes the following steps one through five: Step 1: Construct a teacher model and a student model, which are based on the same defect detection network structure.

[0040] Please refer to Figure 2 In this embodiment, both the teacher model and the student model use the same defect detection network GFLv1, and both use ResNet50 as the backbone network and FPN as the neck network.

[0041] The backbone network employs a ResNet50 convolutional neural network to extract multi-scale deep semantic features from the input defect image. ResNet50, through its residual connection structure, effectively mitigates the vanishing gradient problem during deep network training, thereby obtaining stable feature representations.

[0042] The neck network employs a feature pyramid network, fusing features at different scales output from the backbone network to obtain a feature map with multi-scale representation capabilities. The FPN, through top-down paths and lateral connections, effectively combines high-level semantic information with low-level fine-grained information, thereby improving the detection capability for defects of different sizes.

[0043] The detection head consists of two parts: a classification head and a regression head. The classification head is used to predict the defect category of the candidate anchor point region, and the regression head is used to predict the defect location coordinates of the candidate anchor point region.

[0044] During the initialization phase, both the teacher model and the student model use randomly initialized weights or pre-trained weights from other datasets as initial parameters, and ensure that their network structures are completely identical, so that a one-to-one distillation constraint can be achieved in the subsequent distillation training process.

[0045] Through the above construction method, after the teacher model has been fully trained on old category defect samples, its network parameters can be used as the initial weight input of the student model, ensuring that the student model can inherit the detection capability of the teacher model in the incremental learning process, and at the same time, the new and old defect categories are optimized through selective distillation mechanism.

[0046] Step 2: Input the old defect category samples and labels into the teacher model for training, and save the trained network parameters as the initial weights for training the student model.

[0047] Regarding the datasets, the datasets used in this embodiment include the publicly available steel defect dataset NEU-DET and the publicly available PCB defect dataset DeepPCB.

[0048] The NEU-DET dataset includes six defect types: crazing, inclusion, patches, pitted surface, rolled-in scale, and scratches. Each defect type contains 300 images, for a total of 1800 defect images. In this embodiment, the first three defect types—crazing, inclusion, and patches—are designated as the old defect types, while the latter three—pitted surface, rolled-in scale, and scratches—are designated as the new defect types. The dataset is divided into training, validation, and test samples in a 7:2:1 ratio. The training samples are further divided into old and new categories, with 630 images of the old category used for training the teacher model and 630 images of the new category used for training the student model.

[0049] The DeepPCB dataset includes six defect types: open, short, mousebite, spur, copper, and pinhole, totaling 1500 defect images and 10013 annotations. In this embodiment, the first three defect categories—open, short, and mousebite—are designated as the old defect categories, while the latter three—spur, copper, and pinhole—are designated as the new defect categories. The dataset is divided into training, validation, and test samples in a 4:1:1 ratio. The annotations in the training samples are further divided according to the old and new categories, with 3575 old category annotations used for training the teacher model and 3094 new category annotations used for training the student model.

[0050] During the teacher model training phase, samples and labels representing the old defect categories from the dataset are input into the teacher model for training. After training, the teacher model can effectively identify the old defect categories, and its network parameters are saved as initial weights for student model training.

[0051] Step 3: During the training of the student model, the following distillation strategies are executed for the outputs of the teacher model and the student model: On the classification head, classification distillation with mean squared error loss is performed based on high-confidence anchor point regions; on the regression head, regression distillation with L2 loss is performed based on high-quality anchor point selection in a two-stage cascade; on the feature pyramid network, feature distillation with KL divergence loss is performed based on an IoU-driven difficulty-aware mechanism.

[0052] Specifically, the student model initializes its weights using the parameters of the teacher model and learns using new defect category samples during training. The training process of the student model includes the following steps S1–S5: S1: Input the new defect category samples into the student model, and obtain the prediction results of the classification head, regression head and FPN through the forward propagation of the backbone network, neck network and detection head, including the category prediction results of the classification head, the bounding box prediction results of the regression head and the intermediate feature representation of the feature pyramid network.

[0053] S2: Simultaneously feed the new defect category samples, which are the same as those in step S1, into the teacher model to obtain the corresponding classification head, regression head, and FPN outputs. Specifically, these are the classification head output, regression head output, and feature representation of the feature pyramid network, which are located at the same structural position as the student model.

[0054] S3: Align the outputs of the teacher model and the student model on the classification head, regression head, and FPN respectively, and design corresponding distillation strategies for each, and calculate the distillation loss of the three distillation strategies.

[0055] S4: The student model's own loss and distillation loss are weighted and combined to form the total loss, and the student model parameters are updated through backpropagation.

[0056] S5: Repeat steps S1–S4 until training converges to obtain a student model capable of simultaneously recognizing both old and new defect categories.

[0057] In step S3, after alignment is completed, three selective distillation strategies are introduced based on different task objectives: high-confidence classification distillation strategy, high-confidence classification distillation strategy, and IoU-driven difficulty-aware feature distillation strategy.

[0058] Specifically, this embodiment addresses the noise interference and knowledge dilution problems inherent in traditional global classification distillation by proposing a high-confidence classification distillation strategy. This strategy filters the output of the teacher model's classification head based on confidence, performing classification distillation only on high-confidence anchor points in semantically confusing regions. This reduces invalid alignment in low-value regions while preserving the discriminative knowledge of the old categories.

[0059] The teacher model has learned background region features during training on old defect samples, and the background regions of both new and old defect samples exhibit similar statistical characteristics. After inheriting the weights from the teacher model, the student model, when given a new defect sample, shows that the old class probabilities and IoUs of the background region anchors are close to 0, resulting in low overall confidence. If such anchors are used in distillation, they will introduce noise and dilute crucial discriminative knowledge. In contrast, in the foreground region, the student model often misclassifies new categories as similar old categories, resulting in high prediction probabilities and IoUs close to 1, leading to higher overall confidence. These high-confidence predictions encode the feature similarity and texture priors between new and old defects. Aligning these responses allows the student model to maintain consistent class boundaries with the teacher model in semantically confusing regions, preserving old knowledge while preventing new category learning from disrupting the original feature space.

[0060] like Figure 3 As shown, the high-confidence classification distillation method is as follows: Output of the teacher model classification head Softmax normalization is performed along the category channel dimension, and the maximum value is taken as the confidence value C for each anchor point:

[0061] Where t is the softening temperature factor; Calculate the mean on the confidence level C distribution. and variance And based on the following formula, an adaptive threshold that dynamically adjusts with sample characteristics is constructed. :

[0062] in The proportionality coefficient is a hyperparameter, 0 < <1; Select those with confidence levels higher than the adaptive threshold. The set of anchor points is used as the classification distillation region, within which the L2 loss between the teacher model classification head and the student model classification head in the old defect category output is calculated, which serves as the classification distillation loss function.

[0063] The classification distillation loss function is:

[0064] In the formula, For the classification distillation loss function, The output of the old defect category in the student model classification head, CR is the classification distillation region.

[0065] The high-confidence classification distillation strategy can align the classification predictions of the teacher model and the student model at high-confidence anchor points, effectively maintain the student model's ability to distinguish old defect categories, and improve the stability of the learning process for new defect categories, thereby alleviating the catastrophic forgetting problem in incremental defect detection.

[0066] To address the issues of high noise interference and low boundary positioning accuracy in traditional regression distillation methods, this embodiment proposes a two-stage cascaded regression distillation strategy. This strategy employs a two-stage screening mechanism of "geometric prior coarse screening + statistical fine screening," combined with IoU-weighted KL divergence distillation, to achieve accurate screening and knowledge transfer of high-quality anchor points. This reduces invalid alignment interference while improving the regression capability of the student model.

[0067] In traditional regression distillation methods, distillation is typically performed on all anchor points generated in the defect image. However, most of these anchor points do not contain the defect target, and distillation of low-IoU anchor points introduces significant noise, interfering with the student model's learning of effective localization information. Furthermore, due to the ambiguity of industrial defect boundaries, directly using L2 loss to align coordinates amplifies small offsets between the teacher and student models at boundary positions through squared errors, leading to gradient instability and compromising training robustness. Therefore, this embodiment combines the probability distribution regression mechanism of GFLv1 with IoU-weighted KL divergence distillation loss in regression distillation, and employs a two-stage screening process to ensure that distillation is performed only on high-value anchor points.

[0068] Specifically, such as Figure 4 As shown, the two-stage cascaded regression distillation method is as follows: For each ground truth bounding box (GT Box), select the Top-K anchor points that are closest to its center to form a candidate anchor point set covering the core region of the defective target; Calculate the mean of the IoU distribution of the candidate anchor point set. and standard deviation And construct an adaptive threshold based on the following formula. :

[0069] in The proportionality coefficient is a hyperparameter, 0 < <1; Select an IoU value higher than the adaptive threshold. The set of anchor points is used as the regression distillation region; Within the regression distillation region, a softmax operation is performed on the output of the teacher model regression head, and a log_softmax operation is performed on the output of the student model regression head. The distillation loss is then calculated based on the IoU-weighted KL divergence.

[0070] The loss function of the regression distillation is:

[0071] In the formula, The loss function for regression distillation, For the output of the regression head of the teacher model, The output of the regression head for the student model. This indicates the number of anchor frames in the regressive distillation region. For the return distillation region, This represents the IoU value between the anchor point and the matched ground truth bounding box. Let represent the KL divergence function, and t represent the softening temperature factor.

[0072] in, It is a standard normalization function that transforms the output of the regression head of the teacher model into a probability distribution, denoted as P, as shown in the following formula:

[0073] Is Taking the logarithm of Q, denoted as Q, the formula is as follows:

[0074] It is the KL divergence of distribution P relative to distribution Q, and the formula is as follows:

[0075] In the above softmax and log_softmax formulas, N represents the total number of anchor points.

[0076] The two-stage cascaded regression distillation strategy can filter out low IoU noise anchors and then perform IoU-weighted KL divergence distillation on high-quality anchors, enabling the student model to fully inherit the probability distribution information of the teacher model under fuzzy boundaries, which significantly improves the positioning accuracy and stability of the student model.

[0077] To address the contradiction between "over-distillation inhibiting the learning of new categories" and "under-distillation leading to the forgetting of old categories" in traditional feature distillation, this embodiment proposes an IoU-driven difficulty-aware feature distillation strategy. This strategy quantifies the regression difficulty of defect targets using IoU values ​​and adaptively allocates distillation resources using a dynamic threshold mechanism. This ensures that more difficult defect targets receive more attention, thus preserving space for learning new defect category features while maintaining features of existing ones, preventing feature space rigidity.

[0078] Unlike methods that rely on a fixed distillation intensity, the IoU-driven difficulty-aware feature distillation strategy can dynamically adjust the allocation of distillation resources based on the learning difficulty of the defective target: when the maximum matching IoU of the target is high, it indicates that the target is easy to regress, so the threshold is increased and only a small number of high-quality anchors are retained for distillation; when the maximum matching IoU of the target is low, it indicates that the target is difficult to regress, so the threshold is decreased and the number of distillation anchors is increased, guiding the student model to pay more attention to the features of difficult samples, thus achieving a biased allocation of distillation resources.

[0079] Specifically, such as Figure 5 As shown, the IoU-driven difficulty-aware feature distillation method is as follows: Calculate the IoU matrix between all anchor boxes and all ground truth boxes, and take the maximum IoU value of the ground truth boxes that match each anchor box to form a vector I, and take the maximum IoU value of all anchor boxes that match each ground truth box to form a vector G; An adaptive threshold vector is constructed based on the following formula. :

[0080] in The proportionality coefficient is a hyperparameter, 0 < <1; ⊗ indicates element-wise multiplication; Select an IoU value higher than the adaptive threshold vector. The set of anchor points is used as the feature distillation region; Within the feature distillation region, the MSE (Mean Squared Error) loss is calculated for the outputs of the teacher model FPN and the student model FPN, and used as the feature distillation loss function.

[0081] The characteristic distillation loss function is:

[0082] In the formula, The characteristic distillation loss function, The output of the teacher model FPN, The output of the student model FPN, This indicates the number of anchor frames in the characteristic distillation region. This is a characteristic distillation region.

[0083] The IoU-driven difficulty-aware feature distillation strategy can dynamically select the most discriminative feature regions, ensuring that limited distillation resources are prioritized for more difficult defect targets. This achieves a balance between preserving old knowledge and learning new knowledge, significantly improving the overall performance of the model in incremental defect detection tasks.

[0084] Step four: The student model is trained using three distillation strategies under the condition of no old defective samples.

[0085] In this embodiment, the training process of the student model is completed without relying on old defect category samples. Specifically, it first utilizes the three distillation strategies described in step three, namely: Categorical distillation: Between the classification heads of the teacher model and the student model, for the high-confidence anchor region, calculate the mean squared error loss of both on the old defect category output; Regression distillation: Based on a two-stage cascaded high-quality anchor selection mechanism, the L2 loss of the bounding box prediction results is calculated between the regression heads of the teacher model and the student model. Feature distillation: The feature outputs of the teacher model and the student model are aligned using an IoU-driven difficulty-aware mechanism and constrained by KL divergence loss.

[0086] During training, the three distillation losses mentioned above are weighted and combined with the student model's own detection losses (including classification loss and regression loss) to form a total loss function. The parameters of the student model are then iteratively updated using the backpropagation algorithm. After several training cycles, once the training loss converges, a student model capable of simultaneously recognizing both old and new defect categories is obtained.

[0087] Step 5: Input the defect sample to be detected into the student model; the student model outputs the defect category and its location coordinates of the defect sample.

[0088] After training is complete, the student model obtained in step four is used for defect detection. Specifically, the defect samples to be detected are input into the student model, which then performs forward inference sequentially through the backbone network (ResNet50), the feature pyramid network (FPN), the classification head, and the regression head.

[0089] The system comprises two parts: a classification head that outputs the classification probability distribution of the defect sample across each defect category, and a regression head that outputs the location coordinates of the defect target in the image. The system determines the defect category of the defect sample based on the classification probability distribution and combines this with the prediction results from the regression head to obtain the corresponding defect target bounding box, thus completing the detection and output of the defect category and its location coordinates.

[0090] To verify the effectiveness of the incremental defect detection method based on selective distillation proposed in this embodiment, the method of this embodiment is compared with joint training methods, fine-tuning methods, and existing incremental defect detection methods FSCE, IKLF, and ERD on the NEU-DET and DeepPCB datasets. The evaluation metrics are mAP_50 (mAP_old) for the old defect category, mAP_50 (mAP_new) for the new defect category, and mAP_50 (mAP_all) for all defect categories. The results are shown in Tables 1 and 2.

[0091] Table 1 Comparison of experimental results on the NEU-DET dataset

[0092] Table 2 Comparison of experimental results for the DeepPCB dataset Joint training, which trains all defect categories on GFLv1 at once, is considered the upper limit for incremental defect detection. Fine-tuning, which trains directly on the old category defect detection model using samples of the new defect category, suffers from catastrophic forgetting and is considered the lower limit for incremental defect detection. As shown in Tables 1 and 2, compared to the fine-tuning method, the method in this embodiment improves performance by 40.9%, 0.9%, and 20.9% on NEU-DET, and by 68.8%, 2.4%, and 35.6% on DeepPCB, respectively. This indicates that the method in this embodiment effectively mitigates the catastrophic forgetting problem. On NEU-DET, the mAP_old of this embodiment is only higher than ERD, but mAP_new and mAP_all are higher than other existing incremental defect detection methods. On DeepPCB, the mAP_old, mAP_new, and mAP_all of this embodiment are all higher than other existing incremental defect detection methods.

[0093] Example 2 This embodiment is an incremental defect detection system based on selective distillation. The system is used to implement the incremental defect detection method as described in Embodiment 1, including: The data input module is used to receive old defect category samples and new defect category samples, and input the samples and their labels into the teacher model and student model; The teacher model module is used to train old defect category samples based on the same defect detection network structure and output the prediction results of the classification head, regression head and feature pyramid network. The student model module, whose initial weights are inherited from the network parameters of the teacher model, is used for training on the detection of new defect category samples; The distillation strategy execution module is used to execute the following distillation strategy based on the outputs of the teacher model and the student model during the student model training process: At the classification head, classification distillation with mean squared error loss is performed based on the high-confidence anchor point region; At the regression head, regression distillation of KL divergence loss is performed based on high-quality anchor point screening in a two-stage cascade. On the feature pyramid network, feature distillation with mean squared error loss is performed based on an IoU-driven difficulty-aware mechanism. The model training module is used to weight and combine the self-detection loss and distillation loss of the student model to form the total loss, and update the parameters of the student model through backpropagation until the training converges. The defect detection module is used to input the defect sample to be detected into the trained student model and output the defect category and its location coordinates of the defect sample.

[0094] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several modifications and improvements without departing from the inventive concept of the present invention, and these all fall within the protection scope of the present invention.

Claims

1. An incremental defect detection method based on selective distillation, characterized in that, Includes the following steps: Construct a teacher model and a student model, which are based on the same defect detection network structure; The old defect category samples and labels are input into the teacher model for training, and the trained network parameters are saved as the initial weights for training the student model. During the training of the student model, the following distillation strategy is performed on the outputs of the teacher model and the student model: For the classification head, classification distillation with mean squared error loss is performed based on the high confidence anchor point region; for the regression head, regression distillation with L2 loss is performed based on the high-quality anchor point selection of the two-stage cascade. On the feature pyramid network, feature distillation using KL divergence loss is performed based on an IoU-driven difficulty-aware mechanism. The student model was trained using three distillation strategies under conditions without old defective samples. The defect sample to be detected is input into the student model, and the student model outputs the defect category and its location coordinates of the defect sample.

2. The method according to claim 1, characterized in that, Both the teacher and student models use the same defect detection network GFLv1, with ResNet50 as the backbone and FPN as the neck network.

3. The method according to claim 2, characterized in that, The training method for the student model is as follows: S1: Input the new defect category samples into the student model to obtain the prediction results of the classification head, regression head, and FPN; S2: Feed the same input samples into the teacher model at the same time to obtain the corresponding classification head, regression head and FPN outputs; S3: Align the outputs of the teacher model and the student model on the classification head, regression head and FPN respectively, and design corresponding distillation strategies for each, and calculate the distillation loss of the three distillation strategies. S4: The student model's own loss and distillation loss are weighted and combined to form the total loss, and the student model parameters are updated through backpropagation; S5: Repeat steps S1–S4 until training converges to obtain a student model capable of simultaneously recognizing both old and new defect categories.

4. The method according to claim 2, characterized in that, The specific method of classification distillation is as follows: Output of the teacher model classification head Softmax normalization is performed on the category channel dimension, and the maximum value is taken as the confidence value C for each anchor point; Calculate the mean on the confidence level C distribution. and variance And construct an adaptive threshold based on the following formula. : in The proportionality coefficient is a hyperparameter, 0 < <1; Select those with confidence levels higher than the adaptive threshold. The set of anchor points is used as the classification distillation region, within which the L2 loss between the teacher model classification head and the student model classification head in the old defect category output is calculated, which serves as the classification distillation loss function.

5. The method according to claim 4, characterized in that, The loss function for the classified distillation is: In the formula, For the classification distillation loss function, The output of the old defect category in the student model classification head, CR is the classification distillation region.

6. The method according to claim 2, characterized in that, The regression distillation method is specifically as follows: For each ground truth bounding box, select the Top-K anchor points that are closest to its center to form a candidate anchor point set; Calculate the mean of the IoU distribution of the candidate anchor point set. and standard deviation And construct an adaptive threshold based on the following formula. : in The proportionality coefficient is a hyperparameter, 0 < <1; Select an IoU value higher than the adaptive threshold. The set of anchor points is used as the regression distillation region; Within the regression distillation region, a softmax operation is performed on the output of the teacher model regression head, and a log_softmax operation is performed on the output of the student model regression head. The distillation loss is then calculated based on the IoU-weighted KL divergence.

7. The method according to claim 6, characterized in that, The loss function of the regression distillation is: In the formula, The loss function for regression distillation, For the output of the regression head of the teacher model, The output of the regression head for the student model. This indicates the number of anchor frames in the regressive distillation region. For the return distillation region, This represents the IoU value between the anchor point and the matched ground truth bounding box. Let represent the KL divergence function, and t represent the softening temperature factor.

8. The method according to claim 2, characterized in that, The specific characteristic distillation method is as follows: Calculate the IoU matrix between all anchor boxes and all ground truth boxes, and take the maximum IoU value of the ground truth boxes that match each anchor box to form a vector I, and take the maximum IoU value of all anchor boxes that match each ground truth box to form a vector G; An adaptive threshold vector is constructed based on the following formula. : in The proportionality coefficient is a hyperparameter, 0 < <1; ⊗ indicates element-wise multiplication; Select an IoU value higher than the adaptive threshold vector. The set of anchor points is used as the feature distillation region; Within the feature distillation region, the MSE loss is calculated for the outputs of the teacher model FPN and the student model FPN, and used as the feature distillation loss function.

9. The method according to claim 8, characterized in that, The characteristic distillation loss function is: In the formula, The characteristic distillation loss function, The output of the teacher model FPN, The output of the student model FPN, This indicates the number of anchor frames in the characteristic distillation region. This is a characteristic distillation region.

10. An incremental defect detection system based on selective distillation, characterized in that, The system is used to implement the incremental defect detection method as described in any one of claims 1-9, including: The data input module is used to receive old defect category samples and new defect category samples, and input the samples and their labels into the teacher model and student model; The teacher model module is used to train old defect category samples based on the same defect detection network structure and output the prediction results of the classification head, regression head and feature pyramid network. The student model module, whose initial weights are inherited from the network parameters of the teacher model, is used for training on the detection of new defect category samples; The distillation strategy execution module is used to execute the following distillation strategy based on the outputs of the teacher model and the student model during the student model training process: At the classification head, classification distillation with mean squared error loss is performed based on the high-confidence anchor point region; At the regression head, regression distillation of KL divergence loss is performed based on high-quality anchor point screening in a two-stage cascade. On the feature pyramid network, feature distillation with mean squared error loss is performed based on an IoU-driven difficulty-aware mechanism. The model training module is used to weight and combine the self-detection loss and distillation loss of the student model to form the total loss, and update the parameters of the student model through backpropagation until the training converges. The defect detection module is used to input the defect sample to be detected into the trained student model and output the defect category and its location coordinates of the defect sample.

Images