Defect automatic detection method and system based on cross-resolution and multilevel knowledge distillation

Abstract

The invention discloses an automatic defect detection method and system based on cross-resolution and multi-level knowledge distillation. The method comprises the following steps: step 1, acquiring an image of an intelligent manufacturing product to be detected; 2, constructing a teacher-student network distillation model for intelligent manufacturing defect detection; 3, training the teacher network model in the step 2 by using the data set obtained in the step 1; 4, training a student network model in the teacher-student network distillation model in a cross-resolution and multi-level manner; and step 5, utilizing the trained student network model to carry out defect detection on a to-be-detected intelligent manufacturing product image. A cross-resolution and multi-level knowledge distillation framework is provided, and a collaborative distillation mechanism of local details and a spatial structure is designed; a morphological self-adaptive semantic distillation method is provided, and the method can be better suitable for dynamic and real-time defect detection of intelligent manufacturing products.

Description

Technical Field

[0001] This invention belongs to the field of automatic detection, and specifically relates to an automatic defect detection method and system based on cross-resolution, multi-level knowledge distillation. Background Technology

[0002] In the field of intelligent manufacturing, product defect detection has become a crucial link in ensuring product quality and improving production efficiency. In recent years, the rapid advancements in deep learning, especially the introduction of Convolutional Neural Networks (CNNs) and Visual Transformers, have significantly improved the feature extraction and recognition capabilities for defect detection. However, compared to natural image recognition tasks, industrial defect detection has significantly different characteristics: defects are often small in scale, irregular in shape, sparsely distributed, and coexist at multiple scales. To accurately capture these subtle flaws, high-precision teacher models typically need to be trained and inferred on high-resolution images to fully leverage their powerful feature representation capabilities. However, these models are complex in structure and computationally expensive, making it difficult to meet the stringent requirements of real-time performance and computational efficiency in practical industrial deployments. In contrast, while lightweight student models are computationally efficient, their representation capabilities are limited under low-resolution input, leading to a significant decrease in detection performance. Therefore, how to achieve efficient and high-fidelity knowledge transfer under cross-resolution conditions has become a core challenge in the current field of surface defect detection.

[0003] Traditional feature distillation methods typically rely on numerical similarity metrics (such as L2 distance or KL divergence) to distill the feature representations of the teacher and student models. However, in cross-resolution distillation scenarios, the feature maps of the teacher and student models exhibit significant differences in both channel dimension (C) and spatial dimensions (H, W). Existing methods often utilize the fixed scale ratio between layers in a Feature Pyramid Network (FPN) to select feature maps with consistent spatial dimensions for distillation across different layers. However, this spatial alignment-based strategy ignores the inconsistency in channel dimension and limits the model's ability to flexibly adapt to multi-resolution inputs, thus weakening its potential in practical applications. More importantly, the multi-scale and complex morphology of defects makes it difficult to fully convey the teacher model's deep understanding of defects at high resolutions using only simple channel or spatial alignment. Summary of the Invention

[0004] To address the above technical issues, this application re-examines the essence of cross-resolution knowledge transfer, arguing that the knowledge of the teacher model at high resolution should be structurally decomposed and transferred in a guided manner so that the low-resolution student model can inherit its detail perception and semantic cognition capabilities layer by layer. This provides an automatic defect detection method and system based on cross-resolution, multi-level knowledge distillation.

[0005] The technical solution provided by this invention is as follows:

[0006] On the one hand, an automatic defect detection method based on cross-resolution, multi-level knowledge distillation includes:

[0007] Step 1: Acquire images of the smart manufacturing products to be inspected, and perform preprocessing and annotation to obtain a dataset;

[0008] Step 2: Construct a teacher-student network distillation model for defect detection in intelligent manufacturing, including a teacher network model and a lightweight student network model. The two models transfer features and detection capabilities through a knowledge distillation mechanism.

[0009] The teacher network model is used to extract and learn complex defect feature information, while the lightweight student network model is used to achieve lightweight and efficient inference while maintaining detection accuracy.

[0010] Step 3: Use the dataset obtained in Step 1 to train the teacher network model in Step 2, and obtain the teacher network model with the best training results.

[0011] Step 4: Training a lightweight student network model in the teacher-student network distillation model across resolutions and levels;

[0012] Using defective images from the dataset, input at high and low resolutions into the optimal teacher network model and the lightweight student network model respectively. Through forward propagation, three-level features are extracted from the feature pyramid network of the optimal teacher network model and the lightweight student network model respectively. Knowledge extraction and knowledge distillation loss are calculated for the two sets of three-level features extracted respectively. The distillation loss is combined with the detection loss of the lightweight student network model itself, and the trained lightweight student network model is obtained through backpropagation algorithm optimization.

[0013] Step 5: Use the trained lightweight student network model to perform defect detection on the images of the smart manufacturing products to be inspected.

[0014] Knowledge distillation loss includes detail distillation loss, structural distillation loss, and semantic distillation loss;

[0015] Furthermore, the three hierarchical features include detail-level features, structural-level features, and semantic-level features.

[0016] Furthermore, the detailed calculation process for distillation loss is as follows:

[0017] Step A1: Perform a linear transformation on the channel space of the student hierarchical features output in the lightweight student network model using a learnable orthogonal channel projection matrix;

[0018] ;

[0019] Among them, Fs This represents the student hierarchy features output in the lightweight student model. It is the student hierarchy feature after channel alignment. This indicates that matrix W is applied to F. s In the channel dimension, W represents a learnable orthogonal projection matrix that satisfies W T W = I, C t ≥ C s I represents the identity matrix, C t Teacher F t The number of channels, C s F represents the original student hierarchy feature s The number of channels, B is the batch size, and H is the number of channels. s W s These represent the width and height of the student hierarchy feature, respectively.

[0020] Step A2: Use bilinear interpolation to align the student hierarchy features after channel alignment. The spatial resolution is adjusted to match the teacher hierarchy feature F output by the optimal teacher network model. t With consistent resolution, interpolated student hierarchy features were obtained. ;

[0021] Step A3: Calculate the loss of detailed features;

[0022] By calculating F t and The pixel-level mean squared error is used to constrain the lightweight student network model to learn detailed features, resulting in the detailed feature distillation loss:

[0023] ;

[0024] in, This represents the loss due to distillation of detailed features, where i represents the index of the batch sample, B is the batch size, and F is the loss due to distillation of detailed features. t,i and Let represent the teacher hierarchy features and student hierarchy features of the i-th sample in the batch, respectively; Denotes the square of the Frobenius norm of the matrix. H t , W t This indicates the width and height of the teacher hierarchy.

[0025] The teacher hierarchy features are obtained directly from the teacher network output, and the student hierarchy features are obtained directly from the lightweight student network output.

[0026] Furthermore, the calculation process for structural distillation loss is as follows:

[0027] Step B1: Transfer the teacher hierarchy feature F t Spatial dimensions from (H) t W t Projecting onto the spatial dimension of student hierarchical characteristics (H) s W s This process sequentially assigns the characteristics F of each teacher level to each channel. t Two-dimensional features Through orthogonal projection Thus, the spatially aligned teacher hierarchy features are obtained. , The calculation formula is as follows:

[0028] ;

[0029] in, and H are the orthogonally constrained learnable mapping matrices in the height and width directions, respectively. t W t and H s W s These represent the width and height of the teacher and student hierarchical features, respectively. ;

[0030] Step B2: Perform an orthogonal transformation on the channel-aligned student hierarchy features within their own space to obtain... This allows us to obtain the best-matching student hierarchy features that are perfectly aligned with the projected teacher hierarchy features. , The calculation formula is as follows:

[0031] ;

[0032] in, yes Single-channel feature map, and Let C be an orthogonal matrix, and let C represent the number of student level feature channels.

[0033] Step B3: Calculation of structural distillation loss:

[0034] ;

[0035] Among them, L structure The structural feature distillation loss is represented by , where i represents the index of the batch sample, and B is the batch size. and These are the student and teacher hierarchical features of the i-th sample in the batch after bidirectional orthogonal mapping.

[0036] Furthermore, the semantic hierarchical feature extraction process introduces deformable convolution for adaptive spatial sampling to obtain semantic target regions of different shapes and scales, thereby achieving region selection.

[0037] ;

[0038] Where F is the input feature map, F' represents the features extracted through deformable convolution, and w i Here, p represents the original sampling position on the input feature map, and Δp is the convolution weight. i This is the sampling offset obtained by the network learning.

[0039] This approach only performs subsequent semantic extraction and distillation operations on two types of core feature maps, namely the feature map F output by the teacher model. t s Feature map F output by the student model s s .

[0040] Furthermore, channel attention and spatial attention are introduced in the semantic-level feature extraction process to achieve adaptive enhancement of feature F' in the channel and spatial dimensions;

[0041] Step C1: The channel attention part obtains the channel description vector through global average pooling and uses two layers of 1×1 convolution to model the non-linear relationship between channels:

[0042] ;

[0043] Where GAP(·) represents global average pooling, δ(·) is the ReLU activation function, σ(·) is the Sigmoid function, and M... c W1 represents the channel attention weights, and W2 represents the first 1×1 convolutional layer and the second 1×1 convolutional layer.

[0044] Step C2: Output M c As channel weights, they are multiplied channel by channel with the original feature map F:

[0045] ;

[0046] Among them, F c This indicates that channel attention weights M have been applied. c The subsequent feature, ⊙, represents the Hada code product;

[0047] The spatial attention component aims to capture the saliency of features at different spatial locations.

[0048] Step C3: For F c Perform max pooling and average pooling along the channel direction respectively to obtain F.max and F avg And spliced ​​together to form F ss :

[0049] ;

[0050] Step C4: Generate spatial attention map M through deformable convolution. s :

[0051] ;

[0052] Where DeformConv2d represents a variable convolution operation;

[0053] Step C5: The final spatial augmentation feature is: ;

[0054] Step C6: Enhance the features The residual is added to the original input feature F, and then fused using a 1×1 convolution, BatchNorm, and SiLU activation function to obtain the final output feature:

[0055] ;

[0056] Among them, W f To merge convolution weights.

[0057] Furthermore, the semantic distillation loss calculation process is as follows:

[0058] A semantic distillation loss function based on the feature similarity matrix is ​​adopted to ensure that the semantic distribution structure of the teacher network and the student network is consistent. The semantic distillation loss is calculated as follows:

[0059] ;

[0060] Among them, L semantic G represents the semantic feature distillation loss, norm() normalizes the feature map, and G... S G T as follows:

[0061] ;

[0062] in, and These are for students and teachers respectively. The flattened long vector has a dimension of B represents the batch size, C, H, and W represent the number of student / teacher characteristic channels, height, and width, respectively, and T represents the transpose symbol.

[0063] This loss minimizes the difference in feature correlation between students and teachers in the semantic space, enabling the student model to learn the category distribution patterns and semantic decision boundaries of the teacher model.

[0064] Furthermore, the teacher network model adopts the YOLOv5x structure, and the lightweight student network model adopts the lightweight YOLOv5n structure.

[0065] Secondly, a detection system employing the aforementioned automatic defect detection method based on cross-resolution, multi-level knowledge distillation includes:

[0066] Image preprocessing module: Acquires images of smart manufacturing products to be detected, performs preprocessing and annotation, and obtains a dataset;

[0067] Distillation Model Construction Module: Constructs a teacher-student network distillation model for defect detection in intelligent manufacturing, including a teacher network model and a student network model, which transfer features and detection capabilities through a knowledge distillation mechanism;

[0068] Teacher network model training module: The teacher network model of the constructed teacher-student network distillation model is trained using the dataset obtained by the image preprocessing module to obtain the teacher network model with the best training results.

[0069] Student Network Model Training Module: The lightweight student network model in the teacher-student network distillation model is trained across resolutions and multiple levels. Defect images from the dataset are input into the best teacher network model and the lightweight student model at different resolutions (high and low). Through forward propagation, three levels of features are extracted from the feature pyramid networks of the best teacher network model and the lightweight student network model respectively. Knowledge extraction and knowledge distillation loss are calculated for the two sets of three-level features extracted respectively. The distillation loss is combined with the detection loss of the lightweight student network model itself, and the trained lightweight student network model is obtained through backpropagation algorithm optimization.

[0070] Detection module: Utilizes a pre-trained lightweight student network model to perform defect detection on images of smart manufacturing products to be inspected.

[0071] Thirdly, a computer storage medium storing a computer program for being programmed or configured by a processor to perform the steps of the above-described automatic defect detection method based on cross-resolution, multi-level knowledge distillation.

[0072] The technical solution of this invention has the following advantages over the prior art:

[0073] 1. A cross-resolution, multi-level knowledge distillation framework is proposed, which is implemented collaboratively by three complementary and different "professional tutor" modules, enabling the student model to still have high-resolution perception and discrimination capabilities under low-resolution input.

[0074] 2. A collaborative distillation method for local details and spatial structure was designed. By using detail tutors and structure tutors, local fidelity and global consistency alignment of cross-resolution features were achieved, which significantly improved the student model's ability to reproduce details and perceive structure under low-resolution input.

[0075] 3. A morphological adaptive semantic distillation method is proposed. By combining a semantic tutor with a feature pyramid and variable convolution, the student model can adaptively aggregate key local information of different morphological defects in a high-level semantic feature space, thereby learning the semantic representation of the specific morphology of the defect.

[0076] 4. Experimental validation was conducted on two public datasets. The results show that the proposed method effectively improves the accuracy of the baseline model (student network model) and outperforms existing methods. Attached Figure Description

[0077] Figure 1 This is a schematic flowchart of the detection method described in the technical solution of the present invention.

[0078] Figure 2(a) shows a visual comparison of the baseline model and the method described in this embodiment on the task of detecting defects on the casting surface.

[0079] Figure 2(b) shows a visual comparison of the baseline model and the method described in this embodiment on the task of detecting surface defects on PCB circuit boards. Detailed Implementation

[0080] The present invention will now be further described in conjunction with the accompanying drawings and examples.

[0081] Example 1

[0082] like Figure 1 As shown, an automatic defect detection method based on cross-resolution, multi-level knowledge distillation includes:

[0083] Step 1: Acquire images of the smart manufacturing products to be inspected, and perform preprocessing and annotation to obtain a dataset;

[0084] Specifically, visual sensors are used to acquire defects that occur during the intelligent manufacturing process, resulting in a set of raw image data. The defective parts in the images are labeled using the dedicated labeling software Labelme, thus obtaining a dataset for training a lightweight defect network. Data augmentation methods such as flipping and scaling are used to enhance the labeled dataset. The dataset is then divided according to a specific ratio and data category distribution to obtain training and testing sets. This experiment was conducted on the casting dataset CSDD and the PCB dataset for verification.

[0085] Step 2: Construct a teacher-student network distillation model for defect detection in intelligent manufacturing, including a teacher network model and a lightweight student network model. The two models transfer features and detection capabilities through a knowledge distillation mechanism.

[0086] The teacher network model is used to extract and learn complex defect feature information, while the lightweight student network model is used to achieve lightweight and efficient inference while maintaining detection accuracy.

[0087] The teacher network model adopts the YOLOv5x structure, and the lightweight student network model adopts the lightweight YOLOv5n structure.

[0088] The teacher network model comprises a backbone network for multi-scale feature extraction, a feature pyramid network (FPN+PAN) for multi-scale feature fusion, and a detection head module for predicting detection results. The teacher network model first extracts multi-scale feature maps through the CSPDarknet backbone structure to enhance the representation of defects of different sizes. Then, it achieves top-down and bottom-up feature fusion through the feature pyramid network, effectively combining high-level semantic information with low-level spatial features. Finally, the detection head module decodes the fused features and outputs the predicted category and location of the defect. The teacher network model has a deep network structure and a large parameter scale, enabling it to learn richer defect feature information.

[0089] The lightweight student network model adopts a lightweight YOLOv5n structure, including a backbone network, a feature pyramid network (FPN+PAN), and a detection head module. The lightweight student network model features a shallower network layer and fewer parameters, reducing computational complexity and storage overhead, and improving inference speed. The student network acquires multi-scale feature representations and detection knowledge from the teacher network model through a hybrid multi-scale knowledge distillation mechanism, thereby achieving lightweight defect detection while maintaining high detection accuracy. Specifically, the student network uses the lightweight CSPDarknet backbone structure to extract multi-scale features, fuses semantic and spatial features through the feature pyramid network, and finally outputs the predicted category and location of the defect target by the detection head module.

[0090] Step 3: Use the dataset obtained in Step 1 to train the teacher network model in Step 2, and obtain the teacher network model with the best training results.

[0091] Based on existing object detection methods, the YOLOv5x network, with its large parameter set and strong detection performance, was selected as the teacher network model for defect detection. Training and testing sets were constructed using a well-annotated defect image dataset for training and performance evaluation of the teacher network model. During training, a gradient optimization algorithm based on backpropagation was used to update the network parameters to improve the model's detection accuracy for different types and scales of defects. After training, the weights of the model that performed best on the test set were saved as a guide model for the student network model's learning in the subsequent knowledge distillation stage.

[0092] Step 4: Training the student network model in the teacher-student network distillation model across resolutions and at multiple levels;

[0093] Defect images from the dataset are input into the optimal teacher network model and the lightweight student network model at high and low resolutions, respectively. Three-level features are extracted from the feature pyramid network of the optimal teacher network model and the student network model through forward propagation. Knowledge extraction and knowledge distillation loss are calculated for the two sets of three-level features extracted respectively. The distillation loss is combined with the detection loss of the lightweight student model itself, and the trained student network model is obtained by optimizing through backpropagation algorithm.

[0094] During the knowledge distillation training phase, training data is simultaneously input into both the teacher network model and the lightweight student network model. The teacher network model employs the pre-trained and optimally performing YOLOv5x model, while the student network model uses a lightweight YOLOv5n model. During training, on the one hand, the loss functions of the student network in defect classification and location prediction are directly calculated to ensure accurate identification of defect categories and locations. On the other hand, the multi-level knowledge distillation loss is obtained by calculating the feature representation differences between the teacher network model and the lightweight student network model at multi-level feature layers, enabling effective transfer of knowledge from the teacher network to the student network. Finally, the classification and localization losses obtained through direct supervision are weighted and fused with the multi-level knowledge loss based on knowledge distillation to form a joint optimization objective. This objective is used to iteratively update the parameters of the student network model, achieving a balance between detection accuracy and model lightweighting, thereby improving its real-time performance and recognition accuracy in intelligent manufacturing defect detection tasks.

[0095] The three levels of features include detail-level features, structural-level features, and semantic-level features.

[0096] The detailed calculation process for distillation loss is as follows:

[0097] Step A1: Perform a linear transformation on the channel space of the student hierarchical features output in the lightweight student network model using a learnable orthogonal channel projection matrix;

[0098] ;

[0099] Among them, F s This represents the student hierarchy features output in the lightweight student network model. It is the student hierarchy feature after channel alignment. This indicates that matrix W is applied to F. s In the channel dimension, W represents a learnable orthogonal projection matrix that satisfies W T W = I, C t ≥ C s I represents the identity matrix, C t Teacher F t The number of channels, C s F represents the original student hierarchy feature s The number of channels;

[0100] Step A2: Use bilinear interpolation to align the student hierarchy features after channel alignment. The spatial resolution is adjusted to match the teacher hierarchy feature F output by the optimal teacher network model. t With consistent resolution, interpolated student hierarchy features were obtained. ;

[0101] ;

[0102] Here, Resize(⋅, size) represents the spatial scale adjustment function, which is implemented through bilinear interpolation. The adjusted student feature map. and correlates with teacher feature map F in all dimensions t Complete alignment. H t , W t This indicates the width and height of the teacher hierarchy.

[0103] Step A3: Calculate the loss of detailed features;

[0104] By calculating F t and The pixel-level mean squared error is used to constrain the lightweight student network model to learn detailed features, resulting in the detailed feature distillation loss:

[0105] ;

[0106] in, This represents the loss due to distillation of detailed features, where i represents the index of the batch sample, B is the batch size, and F is the loss due to distillation of detailed features. t,i and Let represent the teacher hierarchy features and student hierarchy features of the i-th sample in the batch, respectively; Let Frobenius norm be the square of the matrix, which is the sum of the squares of all elements in the feature map. By normalizing the total number of elements, the loss value becomes independent of the feature map size.

[0107] The calculation process for structural distillation loss is as follows:

[0108] The first step in structural knowledge distillation is structural representation and feature alignment. Structural distillation, based on channel alignment completed by detail distillation, aims to achieve high-dimensional feature matching between teacher and student networks at the spatial structure level, thereby enhancing the student network's deep learning ability of target spatial layout, contextual patterns, and structural relationships.

[0109] Step B1: Transfer the teacher hierarchy feature F t Spatial dimensions from (H) t W t Projecting onto the spatial dimension of student hierarchical characteristics (H) s W s This process sequentially assigns the characteristics F of each teacher level to each channel. t Two-dimensional features Through orthogonal projection Thus, the spatially aligned teacher hierarchy features are obtained. ;

[0110] ;

[0111] in, and H are the orthogonally constrained learnable mapping matrices in the height and width directions, respectively. t W t and H s W s The width and height of the teacher and student features are represented respectively; this transformation yields the spatially aligned teacher feature map. .

[0112] Step B2: Perform an orthogonal transformation on the channel-aligned student hierarchy features within their own space to obtain... This allows us to obtain the best-matching student hierarchy features that are perfectly aligned with the projected teacher hierarchy features. , The calculation formula is as follows:

[0113] ;

[0114] in, yes Single-channel feature map, and Let C be an orthogonal matrix, and let C represent the number of student feature channels. This process can be regarded as a "rotation and alignment" of the student feature space, and finally obtain the feature map corresponding to the fully aligned student hierarchical features.

[0115] Step B3: Calculation of structural distillation loss:

[0116] ;

[0117] Among them, L structure The structural feature distillation loss is represented by , where i represents the index of the batch sample, and B is the batch size. and These are the student and teacher hierarchical features of the i-th sample in the batch after bidirectional orthogonal mapping.

[0118] The semantic hierarchical feature extraction process introduces deformable convolution for adaptive spatial sampling to obtain semantic target regions of different shapes and scales, thereby achieving region selection.

[0119] ;

[0120] Where F is the input feature map, F' represents the features extracted through deformable convolution, and w i Here, p represents the original sampling position on the input feature map, and Δp is the convolution weight. i This is the sampling offset obtained by the network learning.

[0121] This approach only performs subsequent semantic extraction and distillation operations on two types of core feature maps, namely the feature map F output by the teacher model. t s Feature map F output by the student model s s .

[0122] In the process of semantic hierarchical feature extraction, channel attention and spatial attention are also introduced to achieve adaptive enhancement of feature F' in the channel and spatial dimensions;

[0123] Step C1: The channel attention part obtains the channel description vector through global average pooling and uses two layers of 1×1 convolution to model the non-linear relationship between channels:

[0124] ;

[0125] Where GAP(·) represents global average pooling, δ(·) is the ReLU activation function, σ(·) is the Sigmoid function, and M... c W1 represents the channel attention weights, and W2 represents the first 1×1 convolutional layer and the second 1×1 convolutional layer.

[0126] Step C2: Output M c As channel weights, they are multiplied channel by channel with the original feature map F, F c This indicates that channel attention weights M have been applied. c Subsequent characteristics:

[0127]

[0128] Among them, F c This indicates that channel attention weights M have been applied. c The subsequent feature, ⊙, represents the Hada code product;

[0129] The spatial attention component aims to capture the saliency of features at different spatial locations.

[0130] This mechanism can adaptively improve the response of semantically salient channels and reduce the influence of redundant feature channels, thereby improving the semantic discriminativeness of features.

[0131] Step C3: For F c Perform max pooling and average pooling along the channel direction respectively to obtain F. max and F avg And spliced ​​together to form F ss :

[0132] ;

[0133] Step C4: Generate spatial attention map M through deformable convolution. s :

[0134] ;

[0135] Where DeformConv2d represents a variable convolution operation;

[0136] Step C5: The final spatial augmentation feature is:

[0137] ;

[0138] Step C6: Enhance the features The residual is added to the original input feature F, and then fused using a 1×1 convolution, BatchNorm, and SiLU activation function to obtain the final output feature:

[0139] ;

[0140] Among them, W f To merge convolution weights.

[0141] The semantic distillation loss calculation process is as follows:

[0142] A semantic distillation loss function based on the feature similarity matrix is ​​adopted to ensure that the semantic distribution structure of the teacher network and the student network is consistent. The semantic distillation loss is calculated as follows:

[0143] ;

[0144] Among them, L semantic G represents the semantic feature distillation loss, norm() normalizes the feature map, and G... S G T as follows:

[0145] ;

[0146] in, and These are for students and teachers respectively. The flattened long vector has a dimension of B represents the batch size, C, H, and W represent the number of student / teacher characteristic channels, height, and width, respectively, and T represents the transpose symbol.

[0147] This loss minimizes the difference in feature correlation between students and teachers in the semantic space, enabling the student model to learn the category distribution patterns and semantic decision boundaries of the teacher model.

[0148] The final total loss includes the student's prediction loss and multi-level knowledge loss (details, semantics, and structure). By combining backpropagation theory and stochastic gradient descent algorithm to update the parameters of the network model, and through continuous iteration and optimization, the lightweight network can achieve efficient and high-performance detection and identification of defects in the intelligent manufacturing process under the constraint of loss.

[0149] Step 5: Using the trained lightweight student network model, defect detection is performed on the image of the intelligent manufacturing product to be detected, as shown in Figures 2(a) and 2(b). By comparing the detection visualization results of the baseline model (undistilled student network model) and the model used by the method described in this embodiment, Figure 2(a) shows the optimized result of surface defect detection for castings, and Figure 2(b) shows the optimized result of surface defect detection for PCB circuit boards, intuitively demonstrating the general effectiveness of the method described in this embodiment in various industrial scenarios. First, in the CSDD metal surface defect detection task (first and second rows), facing background interference from high reflectivity and complex textures, the method described in this embodiment demonstrates superior feature extraction capabilities, enabling more accurate and stable localization of "Scratches" (marks left after scratching the workpiece surface) and "Spots" (point defects of different colors or textures appearing on the workpiece surface), effectively improving the detection effect of fine and densely distributed defects. Secondly, in the PCB defect detection task (lines 3 and 4), the advantages of the method described in this embodiment are further demonstrated by a significant increase in confidence and a decrease in the false negative rate: not only is the confidence of "missing hole" (a defect where a hole that should have been present on the workpiece was not machined) increased from 0.67 to 0.85, but the confidence of "short" defects is also significantly increased from 0.33, which was close to being missed, to over 0.72. Furthermore, it successfully captured "spur" defects (burrs, extra sharp protrusions on the edges of the workpiece after machining), which were completely missed by the baseline model. These results fully demonstrate that the strategy adopted by the method described in this embodiment can effectively enhance the model's feature discrimination ability for difficult samples, achieving a dual improvement in localization accuracy and recall across different datasets.

[0150] Example 2

[0151] A detection system employing the aforementioned automatic defect detection method based on cross-resolution, multi-level knowledge distillation includes:

[0152] Image preprocessing module: Acquires images of smart manufacturing products to be detected, performs preprocessing and annotation, and obtains a dataset;

[0153] Distillation Model Construction Module: Constructs a teacher-student network distillation model for defect detection in intelligent manufacturing, including a teacher network model and a student network model, which transfer features and detection capabilities through a knowledge distillation mechanism;

[0154] Teacher network model training module: The teacher network model of the constructed teacher-student network distillation model is trained using the dataset obtained by the image preprocessing module to obtain the teacher network model with the best training results.

[0155] Student Network Model Training Module: The lightweight student network model in the teacher-student network distillation model is trained across resolutions and multiple levels. Defect images from the dataset are input into the best teacher network model and the lightweight student model at different resolutions. Through forward propagation, three levels of features are extracted from the feature pyramid networks of the best teacher network model and the student network model respectively. Knowledge extraction and knowledge distillation loss are calculated for the two sets of three-level features extracted respectively. The distillation loss is combined with the detection loss of the student network model itself, and the trained lightweight student network model is obtained through backpropagation algorithm optimization.

[0156] Detection module: Utilizes a pre-trained lightweight student network model to perform defect detection on images of smart manufacturing products to be inspected.

[0157] In this embodiment, the specific implementation methods of each module can be referred to the description of the foregoing method embodiments, and will not be repeated here. Furthermore, the above modules are functional modules, and can be implemented in hardware or software.

[0158] Example 3

[0159] The present invention provides a computer storage medium storing a computer program, which is used to be programmed or configured by a processor to perform the steps of the automatic defect detection method based on cross-resolution, multi-level knowledge distillation.

[0160] Please refer to the explanation of the method above for the specific implementation process of each step.

[0161] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the hardware and software device described in any of the foregoing embodiments, such as the hard drive or memory of the controller. The readable storage medium can also be an external storage device of the controller, such as a plug-in hard drive, Smart MediaCard (SMC), Secure Digital (SD) card, or Flash Card equipped on the controller. Further, the readable storage medium can include both internal storage units and external storage devices of the controller. The readable storage medium is used to store the computer program and other programs and data required by the controller. The readable storage medium can also be used to temporarily store data that has been output or will be output.

[0162] Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned readable storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0163] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application refers to flowchart illustrations and / or instructions executed by a processor of a method, apparatus (system), and computer program product according to embodiments of this application to create means for implementing the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be stored in a computer-readable storage medium capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means that implement the functions specified in one or more flowchart illustrations and / or one or more block diagrams. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, such that the instructions, which execute on the computer or other programmable apparatus, provide steps for implementing the functions specified in one or more flowcharts and / or one or more blocks of a block diagram.

[0164] It should be emphasized that the examples described in this invention are illustrative rather than limiting. Therefore, this invention is not limited to the examples described in the specific embodiments. Any other embodiments derived by those skilled in the art based on the technical solutions of this invention, without departing from the spirit and scope of this invention, whether modifications or substitutions, are also within the protection scope of this invention.

Claims

1. A defect automatic detection method based on cross-resolution, multi-level knowledge distillation, characterized in that, The method comprises the following steps: Step 1: obtaining an intelligent manufacturing product image to be detected, and pre-processing and labeling to obtain a data set; Step 2: constructing a teacher-student network distillation model for intelligent manufacturing defect detection, including a teacher network model and a lightweight student network model, both of which realize the transmission of features and detection capabilities through a knowledge distillation mechanism; Step 3: training the teacher network model in step 2 using the data set obtained in step 1 to obtain a teacher network model with the best training result; Step 4: training the lightweight student network model in the teacher-student network distillation model across different resolutions and multiple levels; Using the defect images in the data set, input the images into the best teacher network model and the lightweight student network model at different resolutions, respectively, extract three levels of features from the feature pyramid network of the best teacher network model and the lightweight student network model through forward propagation, respectively, perform knowledge extraction and knowledge distillation loss calculation on the two groups of three levels of features extracted, combine the distillation loss with the detection loss of the lightweight student network model itself, and optimize the training completed lightweight student network model through a back propagation algorithm; Step 5: using the trained lightweight student network model to detect defects in the intelligent manufacturing product image to be detected.

2. The method of claim 1, wherein, The three levels of features include detail level features, structure level features and semantic level features.

3. The method of claim 2, wherein, The detail distillation loss calculation process is as follows: Step A1: linearly transforming the channel space of the student level feature output from the lightweight student network model using a learnable positive channel projection matrix; ； wherein F s represents the output of the student level feature in the lightweight student neural network model, is the student level feature after channel alignment, represents the matrix W acting on F s on the channel dimension, W represents a learnable orthogonal projection matrix, satisfying W T W = I, C t ≥ C s , I represents a unit matrix, C t represents the channel number of the teacher F t , C s represents the channel number of the original student level feature F s , T represents a matrix transpose symbol, B is a batch size, H s , W s respectively represent the width and height of the student level feature; Step A2: Use bilinear interpolation to align the student hierarchy features after channel alignment. The spatial resolution is adjusted to match the teacher hierarchy feature F output by the optimal teacher network model. t With consistent resolution, interpolated student hierarchy features were obtained. ; Step A3: detail feature loss calculation; The F t With The pixel-level mean square error between the two is used to constrain the lightweight student network model to learn the detail features, and the detail feature distillation loss is obtained: ； where, denotes the detail feature distillation loss, i denotes the index of the batch sample, F t,i and denotes the teacher-level feature and the student-level feature of the i-th sample in the batch, respectively; denotes the square of the Frobenius norm of a matrix, H t , W t denotes the width and height of the teacher-level feature.

4. The method of claim 2, wherein, The structure distillation loss calculation process is as follows: Step B1: Project the spatial dimension of the teacher-level feature F t from (H t , W t ) to the spatial dimension of the student-level feature (H s , W s ), which processes each two-dimensional feature in each teacher-level feature F t channel by channel to obtain the spatially aligned teacher-level feature F by orthogonal projection to , , The calculation formula is as follows: ； wherein, and are orthogonal constrained learnable mapping matrices in height and width directions, respectively, t , W t and H s , W s represent the width and height of the teacher and student level features, respectively, ; Step B2: The student-level features aligned with the channel are once orthogonally transformed in their own space to get Thus, the best matching student-level features completely aligned with the projected teacher-level features are obtained The calculation formula is as follows:​ ； wherein, is a single-channel feature map of and is a standard orthogonal matrix, and C represents the number of student-level feature channels. Step B3: structure distillation loss calculation: ； where L structure represents the structural feature distillation loss, i represents the index of the batch sample, B is the batch size, and are the student and teacher level features of the i-th sample in the batch after bidirectional orthogonal mapping, respectively.

5. The method of claim 2, wherein, The semantic level feature extraction process uses deformable convolution for adaptive spatial sampling to obtain semantic target regions of different shapes and scales, and realizes region selection: ； wherein F is an input feature map, F’ represents a feature extracted by deformable convolution, w i is a convolution weight, p represents an original sampling position on the input feature map, Δp i is a sampling offset learned by the network.

6. The method of claim 5, wherein, Channel attention and spatial attention are also introduced in the semantic level feature extraction process to realize adaptive enhancement of the feature F' in the channel and spatial dimensions; Step C1: the channel attention part obtains a channel description vector through global average pooling, and realizes nonlinear relationship modeling between channels using two 1x1 convolution layers: ； where GAP(·) denotes the global average pooling, δ(·) is the ReLU activation function, σ(·) is the Sigmoid function, M c denotes the channel attention weight, W1 denotes the first 1x1 convolutional layer, and W2 denotes the second 1x1 convolutional layer. Step C2: The output M c As channel weights, multiply with the original feature map F channel-wise: ； wherein F c represents the channel attention weight M c the features after, ⊙ represents the Hadamard product; Step C3: F c Max-pooling and average-pooling are performed along the channel direction respectively to obtain F max and F avg , and are spliced to form F ss : ； Step C4: Generating spatial attention map M by deformable convolution s : ； Wherein, DeformConv2d represents a deformable convolution operation; Step C5: The final spatially enhanced feature is: ; Step C6: The enhanced features The final output features are obtained by adding the residual to the original input features F, fusing through 1x1 convolution, BatchNorm, and SiLU activation function. ； where W f is the fused convolution weight.

7. The method of claim 6, wherein, The semantic distillation loss calculation process is as follows: A semantic distillation loss function based on a feature similarity matrix is used to constrain the semantic distribution structure of the teacher network and the student network to be consistent, and the semantic distillation loss is calculated as follows: ； wherein L semantic represents the semantic feature distillation loss, norm() is the normalization of the feature map, G S , G T is as follows: ； in, and These are for students and teachers respectively. The flattened long vector has a dimension of B represents the batch size, C, H, and W represent the number of student / teacher feature channels, height, and width, respectively, and T represents the transpose symbol.

8. The method of claim 1, wherein, The teacher network model adopts a YOLOv5x structure, and the lightweight student network model adopts a lightweight YOLOv5n structure.

9. A detection system using the defect automatic detection method based on cross-resolution, multi-level knowledge distillation according to any one of claims 1-8, characterized in that, The method comprises the following steps: An image preprocessing module is configured to obtain an intelligent manufacturing product image to be detected, and pre-process and label the image to obtain a data set; A distillation model construction module is configured to construct a teacher-student network distillation model for intelligent manufacturing defect detection, including a teacher network model and a student network model, both of which realize the transmission of features and detection capabilities through a knowledge distillation mechanism; The teacher network model training module trains the teacher network model of the constructed teacher-student network distillation model by using the data set obtained by the image preprocessing module, and obtains a teacher network model with the best training result; The student network model training module trains the lightweight student network model in the teacher-student network distillation model across resolutions and multiple levels, inputs defect images in the data set to the best teacher network model and the lightweight student model at different resolutions respectively, extracts three levels of features from the feature pyramid network of the best teacher network model and the lightweight student network model respectively through forward propagation, performs knowledge extraction and knowledge distillation loss calculation on the two groups of three levels of features extracted respectively, combines the distillation loss with the detection loss of the lightweight student network model itself, and optimizes the trained lightweight student network model through a back propagation algorithm. The detection module uses the trained lightweight student network model to detect defects in the intelligent manufacturing product images to be detected.

10. A computer storage medium having stored therein a computer program, characterized in that, The computer program is used for programming or configuring a processor to execute the steps of the defect automatic detection method based on cross-resolution and multi-level knowledge distillation in any one of claims 1-8.

Images