A surface defect detection method and system based on feature focusing refinement

Abstract

This invention discloses a surface defect detection method and system based on feature focusing and refinement, belonging to the field of defect detection technology. The method includes: a target feature focusing and refinement defect detection model comprising a target edge focusing feature extraction network, a target reuse and fusion semantic network, a target semantic-channel-space weighted refinement network, and a target detection layer; the target edge focusing feature extraction network outputs multiple multi-scale focusing feature vectors of defect edges based on the image to be detected; each multi-scale focusing feature vector of defect edges is input into the target reuse and fusion semantic network to generate multiple reuse and fusion semantic features; the target semantic-channel-space weighted refinement network determines multiple semantic-channel-space weighted refinement features based on each reuse and fusion semantic feature; and the target detection layer outputs the defect detection result of the image to be detected based on each semantic-channel-space weighted refinement feature. This improves the accuracy of small target defect detection.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a surface defect detection method and system based on feature focusing refinement. Background Technology

[0002] With the deep integration of new-generation information technology and manufacturing, the focus is gradually shifting from quantity expansion to quality improvement. In particular, industries such as clothing and bags have relatively high requirements for product surface quality, and surface defect detection is an important part of quality control.

[0003] Currently, existing surface defect detection methods typically employ deep learning detection techniques based on the YOLO model. By setting an attention mechanism in the backbone of the YOLO model, key channels and key spatial locations in the feature map are extracted and enhanced. Then, based on the feature information contained in the key channels and key spatial locations, defects in the fused feature map are identified. However, the use of the attention mechanism does not remove redundant features, resulting in low detection accuracy for small target defects. Summary of the Invention

[0004] This invention provides a surface defect detection method and system based on feature focusing refinement, which solves the technical problem of low detection accuracy of existing surface defect detection methods for small target defects.

[0005] The first aspect of this invention provides a surface defect detection method based on feature focusing and refinement, comprising:

[0006] The image to be detected is acquired and input into a preset target feature focusing and refining defect detection model. The target feature focusing and refining defect detection model includes a target edge focusing feature extraction network, a target reuse and fusion semantic network, a target semantic-channel-spatial weighted refining network, and a target detection layer.

[0007] The target edge focusing feature extraction network is used to extract multi-scale features from the image to be detected, and outputs multiple multi-scale focusing feature vectors of defect edges.

[0008] The multi-scale focusing feature vectors of each defect edge are input into the target reuse and fusion semantic network for semantic enhancement, thereby generating multiple reuse and fusion semantic features.

[0009] The target semantic-channel-spatial weighted refinement network is used to filter the features of each reused and fused semantic feature, thereby determining multiple semantic-channel-spatial weighted refinement features.

[0010] The target detection layer performs defect localization and detection based on the semantic-channel-spatial weighted refinement features, and outputs the defect detection results of the image to be detected.

[0011] Optionally, the target edge focusing feature extraction network includes convolutional layers, convolutional recursive fusion layers, and spatial pyramid pooling layers; the feature extraction of the image to be detected by the target edge focusing feature extraction network, outputting multiple defect edge multi-scale focusing feature vectors, includes:

[0012] After the image to be detected is input into the convolutional layer for convolution operation, convolutional recursive feature extraction is performed through two cascaded convolutional recursive fusion layers to obtain the first defect edge multi-scale focusing feature vector.

[0013] A convolutional recursive fusion layer is used to process the first defect edge multi-scale focusing feature vector to generate the second defect edge multi-scale focusing feature vector.

[0014] After performing feature mapping on the second defect edge multi-scale focusing feature vector through a convolutional recursive fusion layer, the input spatial pyramid pooling layer performs multi-scale pooling processing, and outputs the third defect edge multi-scale focusing feature vector.

[0015] Optionally, the convolutional recursive fusion layer includes a convolutional layer and a recursive convolutional enhancement layer. The recursive convolutional enhancement layer includes a recursive gated convolutional sub-layer, a feature segmentation sub-layer, and a recursive bottleneck block. The recursive bottleneck block includes multiple cascaded recursive gated convolutional sub-layers. The processing procedure of the convolutional recursive fusion layer includes:

[0016] After performing convolution calculations on the convolutional input features input to the convolutional recursive fusion layer, recursive feature extraction is performed through a recursive gated convolutional sub-layer to generate intermediate convolutional features.

[0017] The convolutional intermediate features are split using a feature segmentation sub-layer, and the first convolutional split feature and the second convolutional split feature are output.

[0018] After the second convolution split feature is input into multiple cascaded recursive gated convolution sub-layers for continuous feature extraction, it is fused with the second convolution split feature to obtain the first convolution fusion feature.

[0019] After extracting features from the first convolutional fusion feature by cascading multiple recursive gated convolutional sub-layers, the features are fused with the first convolutional fusion feature to determine the second convolutional fusion feature.

[0020] After concatenating the first convolution split feature, the first convolution fusion feature, and the second convolution fusion feature through channels, they are input into a recursive gated convolution sub-layer for convolution processing to generate convolution output features.

[0021] Optionally, the target reuse fusion semantic network includes an attention layer, a depthwise separable convolutional layer, and residual blocks; the step of inputting the multi-scale focused feature vectors of each defect edge into the target reuse fusion semantic network for semantic enhancement generates multiple reuse fusion semantic features, including:

[0022] The multi-scale focusing feature vectors of each defect edge are respectively input into the cascaded attention layer and depth-separable convolutional layer for channel space feature extraction and fusion, corresponding to the first hemispherical perceptual embedding feature, the second hemispherical perceptual embedding feature and the third hemispherical perceptual embedding feature;

[0023] After upsampling the third hemispherical perception embedding feature, it is fused with the second hemispherical perception embedding feature to generate the first semantic fusion feature;

[0024] After upsampling the first semantic fusion feature, it is fused with the first hemispherical perception embedding feature to output the second semantic fusion feature.

[0025] By extracting features from the first semantic fusion feature and the second semantic fusion feature using residual blocks, residual features and the first reused fusion semantic feature are obtained respectively.

[0026] After downsampling the first multiplexed fusion semantic feature, it is fused with the residual feature, and then the feature is processed through the residual block to output the second multiplexed fusion semantic feature.

[0027] After downsampling using the residual features, the features are fused with the third tomographic perception embedding features, and then feature extraction is performed via the residual blocks to output the third reused fusion semantic features.

[0028] Optionally, the target semantic-channel-spatial weighted refinement network includes a global average pooling layer, a global max pooling layer, a Conv_Silu layer, a Sigmoid activation function layer, and a Softmax activation function layer; the processing procedure of the target semantic-channel-spatial weighted refinement network includes:

[0029] The reused and fused semantic features input to the target semantic-channel-space weighted refinement network are subjected to average pooling and max pooling respectively through a global average pooling layer and a global max pooling layer, generating average pooling features and max pooling features accordingly.

[0030] After performing convolution operations on the average pooling features and the max pooling features through the Conv_Silu layer, feature fusion is performed to output channel features;

[0031] The channel weights of the channel features are extracted using a Sigmoid activation function layer, and the channel weights are multiplied element-wise with the reused fusion semantic features to obtain channel-weighted features;

[0032] The spatial weights of the reused and fused semantic features are extracted based on the Softmax activation function layer, and the spatial weights are multiplied element-wise with the reused and fused semantic features to output the spatial weighted features.

[0033] The channel-weighted features and the spatial-weighted features are fused to generate semantic-channel-spatial-weighted refined features.

[0034] Optionally, the target feature focuses on refining the training process of the defect detection model, including:

[0035] Obtain low-resolution images with defect annotations;

[0036] The initial edge focusing feature extraction network of the initial feature focusing refinement defect detection model is trained locally using the defect-annotated low-resolution image. When the first loss function value converges, the intermediate feature focusing refinement defect detection model is output.

[0037] The intermediate feature-focused refinement defect detection model is trained on the low-resolution image with the defect annotation until the second loss function value converges, thus determining the target feature-focused refinement defect detection model.

[0038] Optionally, the step of using the low-resolution image with defect annotations to train the initial edge focusing feature extraction network of the initial feature focusing refinement defect detection model locally, and outputting an intermediate feature focusing refinement defect detection model when the first loss function converges, includes:

[0039] In the initial edge focus feature extraction network of the initial feature focus refinement defect detection model, after the convolutional layer performs convolutional processing on the low-resolution image of the defect annotation, convolutional recursive feature extraction is performed based on two cascaded convolutional recursive fusion layers to generate training low-level features.

[0040] The training low-level features are processed continuously by two convolutional recursive fusion layers cascaded in the initial edge focusing feature extraction network to obtain the training high-level features.

[0041] The trained low-level features and the trained high-level features are input into a preset defect edge enhancement deep feature aggregation network for feature fusion enhancement, and the trained high-resolution image is output.

[0042] The first loss function value is calculated based on the training high-resolution image and the defect-annotated high-resolution image corresponding to the defect-annotated low-resolution image;

[0043] The model parameters in the initial edge focusing feature extraction network are iteratively optimized according to the first loss function value until the first loss function value converges, thus obtaining the intermediate feature focusing refinement defect detection model.

[0044] Optionally, the defect edge enhancement deep feature aggregation network includes an encoder and a decoder. The encoder includes an upsampling layer and a convolutional layer with an activation function, and the decoder includes a convolutional layer with an activation function and an image reconstruction layer. The step of inputting the trained low-level features and the trained high-level features into a preset defect edge enhancement deep feature aggregation network for feature fusion enhancement, and outputting a training high-resolution image, includes:

[0045] The training low-level features are upsampled by an upsampling layer to obtain the first encoded features;

[0046] The training high-level features are mapped using a convolutional layer with an activation function to generate a second encoded feature;

[0047] The first encoded feature and the second encoded feature are concatenated by channels to output the image encoded feature;

[0048] The first decoded feature is obtained by continuously extracting features from the image coding features through multiple convolutional layers with activation functions.

[0049] An image reconstruction layer is used to perform image super-resolution reconstruction based on the first decoding features to generate a training high-resolution image.

[0050] A second aspect of the present invention provides a surface defect detection system based on feature focusing refinement, comprising:

[0051] The feature input module is used to acquire the image to be detected and input the image to be detected into a preset target feature focusing and refining defect detection model. The target feature focusing and refining defect detection model includes a target edge focusing feature extraction network, a target reuse and fusion semantic network, a target semantic-channel-spatial weighted refining network, and a target detection layer.

[0052] The feature extraction module is used to perform multi-scale feature extraction on the image to be detected through the target edge focusing feature extraction network, and output multiple defect edge multi-scale focusing feature vectors;

[0053] The feature enhancement module is used to input the multi-scale focused feature vectors of each defect edge into the target reuse and fusion semantic network for semantic enhancement, thereby generating multiple reuse and fusion semantic features.

[0054] The feature refinement module is used to perform feature filtering on each of the multiplexed and fused semantic features using the target semantic-channel-space weighted refinement network, and to determine multiple semantic-channel-space weighted refinement features accordingly.

[0055] The defect detection module is used to perform defect localization and detection based on the semantic-channel-spatial weighted refinement features of the target detection layer, and output the defect detection result of the image to be detected.

[0056] A computer device provided in a third aspect of the present invention includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor performs the steps of the surface defect detection method based on feature focusing refinement as described in any of the preceding claims.

[0057] As can be seen from the above technical solutions, the present invention has the following advantages:

[0058] The above-mentioned scheme of the present invention inputs the image to be detected into the target feature focusing and refining defect detection model. First, it focuses on small target features through the target edge focusing feature extraction network to extract multi-scale features from the image to be detected. Then, it uses the target reuse fusion semantic network to better capture effective information about defect features. Next, it uses the target semantic-channel-spatial weighted refining network to filter redundant features. Finally, it outputs to the target detection layer to locate the defect region, thereby improving the detection accuracy of small target defects. Attached Figure Description

[0059] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0060] Figure 1 A flowchart illustrating the steps of a surface defect detection method based on feature focusing and refinement provided in this embodiment of the invention;

[0061] Figure 2 A framework diagram of the feature-focused refinement defect detection model provided in an embodiment of the present invention;

[0062] Figure 3 This is a schematic diagram of the recursive convolution enhancement layer provided in an embodiment of the present invention;

[0063] Figure 4 A schematic diagram of the structure of the defect edge enhancement depth feature aggregation network provided in an embodiment of the present invention;

[0064] Figure 5 This is a schematic diagram of the structure of the reuse and fusion semantic network provided in an embodiment of the present invention;

[0065] Figure 6 This is a schematic diagram of the semantic-channel-spatial weighted refinement network provided in an embodiment of the present invention;

[0066] Figure 7 This is a structural block diagram of a surface defect detection system based on feature focusing refinement provided in an embodiment of the present invention. Detailed Implementation

[0067] This invention provides a surface defect detection method and system based on feature focusing and refinement, which solves the technical problem that existing surface defect detection methods have low accuracy in detecting small target defects.

[0068] To make the objectives, features, and advantages of this invention more apparent and understandable, the technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described below are only some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0069] Please see Figure 1 , Figure 1 The flowchart illustrates the steps of a surface defect detection method based on feature focusing and refinement, as provided in this embodiment of the invention.

[0070] This invention provides a surface defect detection method based on feature focusing and refinement, comprising:

[0071] Step 101: Obtain the image to be detected and input the image to be detected into the preset target feature focusing and refining defect detection model. The target feature focusing and refining defect detection model includes a target edge focusing feature extraction network, a target reuse fusion semantic network, a target semantic-channel-spatial weighted refining network, and a target detection layer.

[0072] It should be noted that the image to be inspected refers to the product image to be inspected for surface defects. The target feature focusing and refinement defect detection model provided in this embodiment is an improvement on the existing YOLOv8, such as... Figure 2 As shown, it includes a target edge focusing feature extraction network, a target reuse fusion semantic network (RBANet), a target semantic-channel-spatial weighted refinement network (PFF), and a target detection layer (Detect).

[0073] Step 102: Extract multi-scale features from the image to be detected using the target edge focusing feature extraction network, and output multiple multi-scale focusing feature vectors of defect edges.

[0074] like Figure 2 As shown, the target edge focusing feature extraction network includes convolutional layers (Conv), convolutional recurrent fusion layers, and spatial pyramid pooling layers (SPPF); step 102 includes the following sub-steps:

[0075] After inputting the image to be detected into the convolutional layer for convolution operation, the convolutional recursive feature extraction is performed through two cascaded convolutional recursive fusion layers to obtain the first defect edge multi-scale focusing feature vector.

[0076] A convolutional recursive fusion layer is used to process the multi-scale focusing feature vector of the first defect edge to generate the multi-scale focusing feature vector of the second defect edge.

[0077] After performing feature mapping on the multi-scale focused feature vector of the second defect edge through a convolutional recursive fusion layer, the input spatial pyramid pooling layer performs multi-scale pooling processing, and outputs the multi-scale focused feature vector of the third defect edge.

[0078] Furthermore, such as Figure 2 and Figure 3 As shown, the convolutional recursive fusion layer includes a convolutional layer (Conv) and a recursive convolutional enhancement layer (GnC2f). The recursive convolutional enhancement layer includes a recursive gated convolutional sub-layer (GnConv), a feature segmentation sub-layer (split), and a recursive bottleneck block (GnDarknetBottleneck). The recursive bottleneck block includes multiple cascaded recursive gated convolutional sub-layers. The processing of the convolutional recursive fusion layer includes:

[0079] After performing convolution calculations on the convolutional input features of the input convolutional recursive fusion layer, recursive feature extraction is performed through a recursive gated convolutional sub-layer to generate intermediate convolutional features.

[0080] A feature segmentation sub-layer is used to split the intermediate features of the convolution, and the first and second convolution split features are output.

[0081] After the second convolution split feature is input into multiple cascaded recursive gated convolution sub-layers for continuous feature extraction, it is fused with the second convolution split feature to obtain the first convolution fused feature.

[0082] After extracting features from the first convolutional fusion feature by cascading multiple recursive gated convolutional sub-layers, the features are fused with the first convolutional fusion feature to determine the second convolutional fusion feature.

[0083] After concatenating the first convolution split feature, the first convolution fusion feature, and the second convolution fusion feature through channels, the input is processed by the recursive gated convolution sub-layer to generate the convolution output feature.

[0084] It should be noted that the target edge focusing feature extraction network is based on CSPDarkNet-53 and introduces a recursively gated convolutional sublayer (GnConv) into the C2f module. The GnC2f structure can extend the second-order interaction in self-attention to any order, reduce the amount of computation, and thus better identify defective product images with small area and low contrast. It can also more efficiently integrate local features and global information, and enhance the network's ability to express contextual relationships.

[0085] Step 103: Input the multi-scale focusing feature vectors of each defect edge into the target reuse and fusion semantic network for semantic enhancement, and generate multiple reuse and fusion semantic features accordingly.

[0086] like Figure 5 As shown, the Target Reuse Fusion Semantic Network (RBANet) includes an attention layer (CBAM), a depthwise separable convolutional layer (DSConv), and residual blocks (ResBlock); step 103 includes the following sub-steps:

[0087] S11. Input the multi-scale focusing feature vectors of each defect edge into the cascaded attention layer and depth separable convolutional layer respectively to extract and fuse channel space features, and obtain the first hemispherical perceptual embedding feature, the second hemispherical perceptual embedding feature and the third hemispherical perceptual embedding feature.

[0088] It should be noted that the multi-scale focusing feature vectors of each defect edge, such as the first, second, and third defect edge multi-scale focusing feature vectors, are defect edge multi-scale focusing feature vectors at different scales. First, each defect edge multi-scale focusing feature vector is used as the attention input feature of the attention layer. The attention layer extracts information sequentially in both the channel and spatial dimensions. Placing the attention layer after the target edge focusing feature extraction network can better capture effective information about defect features, and improve the utilization rate of defect locations by readjusting the channel and spatial features. The specific process of the attention layer is determined by the following formula:

[0089] ;

[0090] ;

[0091] In the formula, To pay attention to input features, To process the channel dimension, This is a global average pooling operation. For max pooling operation, It is a multilayer perceptron. It is the sigmoid activation function. In order to process spatial dimensions, It is a 7x7 convolution. These are the features after processing along the channel dimension;

[0092] Then, the attention output features of the attention layer are passed through a depthwise separable convolutional layer (DSConv) to further fuse the features while reducing the number of convolution parameters.

[0093] S12. After upsampling the third-level perceptual embedding features, perform feature fusion with the second-level perceptual embedding features to generate the first semantic fusion feature.

[0094] S13. After upsampling the first semantic fusion feature, perform feature fusion with the first hemispherical perception embedding feature to output the second semantic fusion feature.

[0095] S14. Extract features from the first semantic fusion feature and the second semantic fusion feature using residual blocks to obtain the residual feature and the first reused fusion semantic feature.

[0096] S15. After downsampling the first multiplexed fusion semantic features, feature fusion is performed with the residual features, and feature processing is performed through the residual block to output the second multiplexed fusion semantic features.

[0097] S16. After downsampling using residual features, feature fusion is performed with the third-level perceptual embedding features, and feature extraction is performed through residual blocks to output the third reuse fusion semantic features.

[0098] It should be noted that the first, second, and third hemispherical perceptual embedding features output by the depthwise separable convolutional layer are subjected to multi-scale feature fusion through an upsampling fusion layer based on the FPN structure. The first and second semantic fusion features output after fusion are subjected to residual processing through a residual block structure based on DSConv, and then processed by a downsampling fusion layer based on the PAN structure. Finally, the hierarchical semantically enhanced reusable fusion semantic features are obtained through a residual block structure based on DSConv.

[0099] Step 104: Use the target semantic-channel-space weighted refinement network to filter the features of each reused and fused semantic feature, and determine multiple semantic-channel-space weighted refinement features accordingly.

[0100] like Figure 6As shown, the Target Semantic-Channel-Spatial Weighted Refinement Network (PFF) includes an Adaptive Average Pooling layer, an Adaptive Maximum Pooling layer, a Conv_Silu layer, a Sigmoid activation function layer, and a Softmax activation function layer. The processing steps of the Target Semantic-Channel-Spatial Weighted Refinement Network include:

[0101] The semantic features of the input target semantic-channel-space weighted refinement network are reused and fused, respectively, and average pooling and max pooling are performed through global average pooling layer and global max pooling layer to generate average pooling features and max pooling features respectively.

[0102] After performing convolution operations on the average pooling features and the max pooling features through the Conv_Silu layer, the features are fused to output channel features;

[0103] The channel weights of the channel features are extracted using a Sigmoid activation function layer, and then the channel weights are multiplied element-wise with the reused and fused semantic features to obtain the channel weighted features.

[0104] Based on the Softmax activation function layer, the spatial weights of the reused and fused semantic features are extracted, and the spatial weights are multiplied element-wise with the reused and fused semantic features to output the spatial weighted features.

[0105] By fusing channel-weighted features with spatial-weighted features, semantic-channel-spatial-weighted refined features are generated.

[0106] It should be noted that the first, second, and third reused fused semantic features are sequentially input into the target semantic-channel-spatial weighted refinement network for processing. First, adaptive channel weights are calculated through channel refinement guided by global average pooling and global max pooling layers. Simultaneously, spatial weights are determined through a Softmax activation function layer. These are then multiplied by the reused fused semantic features to filter out interfering features. The specific process includes:

[0107] ;

[0108] ;

[0109] ;

[0110] In the formula, For channel weights, It is the sigmoid activation function. To reuse and fuse semantic features, After global average pooling, convolution and the SiLu activation function are applied. After max pooling, convolution and the SiLu activation function are applied. Spatial weights, The Softmax activation function is used. It is a semantic-channel-space weighted refinement feature.

[0111] Step 105: Defect localization and detection are performed by the target detection layer based on the semantic-channel-spatial weighted refinement features, and the defect detection results of the image to be detected are output.

[0112] It should be noted that the defects are located and marked in the defect detection results by edge focusing prediction boxes, while the target detection layer uses the original YOLOv8 Detect detection layer for detection. For specific application process, please refer to existing technologies, which will not be elaborated here.

[0113] Furthermore, the target feature-focused refinement defect detection model provided in this embodiment is determined through phased training after improving the existing YOLOv8 to obtain an initial feature-focused refinement defect detection model. The initial feature-focused refinement defect detection model includes an initial edge-focusing feature extraction network, an initial reuse fusion semantic network (RBANet), an initial semantic-channel-spatial weighted refinement network (PFF), and an initial detection layer (Detect). In specific implementation, the training process of the target feature-focused refinement defect detection model includes:

[0114] S31. Obtain a low-resolution image of the defect annotation.

[0115] It should be noted that the low-resolution image for defect annotation refers to an image in which the defect has been annotated and is of low resolution. In practice, defect annotation can be represented by marking the coordinates, length and width of the defect in the image using a rectangular box.

[0116] Furthermore, before step S31, defect annotation training images can be obtained first. After preprocessing the defect annotation training images, low-resolution defect annotation images for input model training can be obtained. The preprocessing can include operations such as image cropping, noise filtering, scale standardization, and contrast adjustment.

[0117] S32. Use the low-resolution image with defect annotation to train the initial edge focus feature extraction network of the initial feature focus refinement defect detection model locally. When the first loss function value converges, output the intermediate feature focus refinement defect detection model.

[0118] like Figure 2 As shown, the process of determining the intermediate feature-focused refinement defect detection model includes:

[0119] S321. In the initial edge focus feature extraction network of the initial feature focus refinement defect detection model, after the convolutional layer performs convolution processing on the low-resolution image of the defect annotation, convolutional recursive feature extraction is performed based on two cascaded convolutional recursive fusion layers to generate training low-level features.

[0120] S322. The training low-level features are continuously processed by two convolutional recursive fusion layers cascaded in the initial edge focusing feature extraction network to obtain the training high-level features.

[0121] S323. Input the low-level training features and high-level training features into the preset defect edge enhancement deep feature aggregation network for feature fusion enhancement, and output the training high-resolution image.

[0122] Furthermore, such as Figure 4 As shown, the defect edge enhancement deep feature aggregation network includes an encoder and a decoder. The encoder includes an upsampling layer and a convolutional layer (CR) with activation functions. The decoder includes a convolutional layer (CR) with activation functions and an image reconstruction layer (EDSR). In a specific implementation, the convolutional layer (CR) with activation functions includes a 3×3 convolution and a ReLU activation function. The process of S323 includes:

[0123] The first encoded feature is obtained by upsampling the low-level training features through an upsampling layer.

[0124] A convolutional layer with an activation function is used to perform feature mapping on the high-level training features to generate a second encoded feature;

[0125] The first and second encoded features are concatenated through channels to output the image encoded features;

[0126] The first decoding feature is obtained by continuously extracting image coding features through multiple convolutional layers with activation functions.

[0127] An image reconstruction layer is used to perform image super-resolution reconstruction based on the first decoding features to generate a training high-resolution image.

[0128] It should be noted that in the image reconstruction layer (EDSR), the first decoded feature is processed by residuals through 32 residual blocks to generate the second decoded feature vector. After feature recombination of the second decoded feature vector through 3×3 convolution, upsampling is performed based on subpixel convolutional layers to obtain a training high-resolution image that is twice the size of the low-resolution image of defect annotation.

[0129] S324. Calculate the first loss function value based on the training high-resolution image and the defect annotation low-resolution image corresponding to the defect annotation high-resolution image.

[0130] It should be noted that the L1 loss is calculated by comparing the high-resolution training image output by the defect edge enhancement deep feature aggregation network with the high-resolution defect annotation image corresponding to the low-resolution defect annotation image. The calculation process includes:

[0131] ;

[0132] In the formula, The first loss function value, The convolution weights are from layer 1 to layer L. For the bias from layer 1 to layer L, The height of the low-resolution image. The width of the low-resolution image. This is the magnification factor. High-resolution images for defect annotation in Pixel value at that location, Label low-resolution images for defects Pixel value at that location, To train high-resolution images in The pixel value at that location.

[0133] S325. Iteratively optimize the model parameters in the initial edge focusing feature extraction network according to the first loss function value until the first loss function value converges, and obtain the intermediate feature focusing refinement defect detection model.

[0134] It should be noted that during the first stage of training the initial feature-focused refinement defect detection model, the model parameters in the initial edge-focused feature extraction network are iteratively updated according to the first loss function value through backpropagation until the first loss function value converges, thereby obtaining the intermediate feature-focused refinement defect detection model. The feature extraction capability of convolutional downsampling in the initial edge-focused feature extraction network is optimized through pre-training, focusing on extracting features of small targets. Since the spatial pyramid pooling layer (SPPF) mainly performs preliminary feature fusion for the last layer of features and does not involve downsampling feature extraction, it does not need to participate in the first stage of training.

[0135] S33. Based on the low-resolution image with defect annotation, train the intermediate feature-focused and refined defect detection model as a whole until the second loss function value converges, and determine the target feature-focused and refined defect detection model.

[0136] It should be noted that during the second stage of model training, low-resolution images with defect annotations are used to train the intermediate feature-focused refined defect detection model as a whole. Finally, the second loss function value is calculated based on the output of the intermediate feature-focused refined defect detection model. The model parameters within the intermediate feature-focused refined defect detection model are iteratively optimized according to the second loss function value until the second loss function value converges to determine the target feature-focused refined defect detection model. In specific implementation, the second loss function value can be the quality focus loss function. The quality focus loss function, used to calculate and obtain the edge focus prediction box, specifically includes the following formula:

[0137] ;

[0138] In the formula, The score is calculated by multiplying the label by the corresponding IOU. The classification score after the sigmoid operation. This is an adjustable parameter.

[0139] In this embodiment of the invention, after the image to be detected is input into the target feature focusing and refining defect detection model, the target edge focusing feature extraction network first focuses on small target features to extract multi-scale features from the image to be detected. Then, based on the target reuse fusion semantic network, effective information about defect features is better captured. Next, the target semantic-channel-spatial weighted refining network is used to filter redundant features. Finally, the output is sent to the target detection layer to locate the defect region, thereby improving the accuracy of small target defect detection.

[0140] Please see Figure 7 , Figure 7 This is a structural block diagram of a surface defect detection system based on feature focusing refinement provided in an embodiment of the present invention.

[0141] This invention provides a surface defect detection system based on feature focusing and refinement, comprising:

[0142] The feature input module 701 is used to acquire the image to be detected and input the image to be detected into a preset target feature focusing and refining defect detection model. The target feature focusing and refining defect detection model includes a target edge focusing feature extraction network, a target reuse and fusion semantic network, a target semantic-channel-spatial weighted refining network, and a target detection layer.

[0143] The feature extraction module 702 is used to extract multi-scale features from the image to be detected through the target edge focusing feature extraction network, and output multiple defect edge multi-scale focusing feature vectors.

[0144] The feature enhancement module 703 is used to input the multi-scale focused feature vectors of each defect edge into the target reuse and fusion semantic network for semantic enhancement, thereby generating multiple reuse and fusion semantic features.

[0145] The feature refinement module 704 is used to perform feature filtering on each reused and fused semantic feature using a target semantic-channel-space weighted refinement network, thereby determining multiple semantic-channel-space weighted refinement features.

[0146] The defect detection module 705 is used to perform defect localization and detection based on semantic-channel-spatial weighted refinement features through the target detection layer, and output the defect detection results of the image to be detected.

[0147] Furthermore, the target edge focusing feature extraction network includes convolutional layers, convolutional recursive fusion layers, and spatial pyramid pooling layers; the feature extraction module 702 is specifically used for:

[0148] After inputting the image to be detected into the convolutional layer for convolution operation, the convolutional recursive feature extraction is performed through two cascaded convolutional recursive fusion layers to obtain the first defect edge multi-scale focusing feature vector.

[0149] A convolutional recursive fusion layer is used to process the multi-scale focusing feature vector of the first defect edge to generate the multi-scale focusing feature vector of the second defect edge.

[0150] After performing feature mapping on the multi-scale focused feature vector of the second defect edge through a convolutional recursive fusion layer, the input spatial pyramid pooling layer performs multi-scale pooling processing, and outputs the multi-scale focused feature vector of the third defect edge.

[0151] Furthermore, the convolutional recursive fusion layer includes convolutional layers and recursive convolutional enhancement layers. The recursive convolutional enhancement layer includes recursive gated convolutional sub-layers, feature segmentation sub-layers, and recursive bottleneck blocks. The recursive bottleneck blocks include multiple cascaded recursive gated convolutional sub-layers. The processing procedure of the convolutional recursive fusion layer includes:

[0152] After performing convolution calculations on the convolutional input features of the input convolutional recursive fusion layer, recursive feature extraction is performed through a recursive gated convolutional sub-layer to generate intermediate convolutional features.

[0153] A feature segmentation sub-layer is used to split the intermediate features of the convolution, and the first and second convolution split features are output.

[0154] After the second convolution split feature is input into multiple cascaded recursive gated convolution sub-layers for continuous feature extraction, it is fused with the second convolution split feature to obtain the first convolution fused feature.

[0155] After extracting features from the first convolutional fusion feature by cascading multiple recursive gated convolutional sub-layers, the features are fused with the first convolutional fusion feature to determine the second convolutional fusion feature.

[0156] After concatenating the first convolution split feature, the first convolution fusion feature, and the second convolution fusion feature through channels, the input is processed by the recursive gated convolution sub-layer to generate the convolution output feature.

[0157] Furthermore, the target reuse fusion semantic network includes attention layers, depthwise separable convolutional layers, and residual blocks; the feature enhancement module 703 is specifically used for:

[0158] The multi-scale focused feature vectors of each defect edge are respectively input into the cascaded attention layer and depth-separable convolutional layer for channel space feature extraction and fusion, which yields the first hemispherical perceptual embedding feature, the second hemispherical perceptual embedding feature and the third hemispherical perceptual embedding feature.

[0159] After upsampling the third-layer perceptual embedding features, feature fusion is performed with the second-layer perceptual embedding features to generate the first semantic fusion feature;

[0160] After upsampling the first semantic fusion feature, it is fused with the first hierarchical perceptual embedding feature to output the second semantic fusion feature.

[0161] By extracting features from the first semantic fusion feature and the second semantic fusion feature using residual blocks, the residual features and the first reused fusion semantic features are obtained respectively.

[0162] After downsampling the first multiplexed fusion semantic features, feature fusion is performed with the residual features, and feature processing is performed through the residual block to output the second multiplexed fusion semantic features.

[0163] After downsampling using residual features, feature fusion is performed with third-level perceptual embedding features, and feature extraction is performed via residual blocks to output third-level reused fusion semantic features.

[0164] Furthermore, the target semantic-channel-spatial weighted refinement network includes a global average pooling layer, a global max pooling layer, a Conv_Silu layer, a Sigmoid activation function layer, and a Softmax activation function layer; the processing procedure of the target semantic-channel-spatial weighted refinement network includes:

[0165] The semantic features of the input target semantic-channel-space weighted refinement network are reused and fused, respectively, and average pooling and max pooling are performed through global average pooling layer and global max pooling layer to generate average pooling features and max pooling features respectively.

[0166] After performing convolution operations on the average pooling features and the max pooling features through the Conv_Silu layer, the features are fused to output channel features;

[0167] The channel weights of the channel features are extracted using a Sigmoid activation function layer, and then the channel weights are multiplied element-wise with the reused and fused semantic features to obtain the channel weighted features.

[0168] Based on the Softmax activation function layer, the spatial weights of the reused and fused semantic features are extracted, and the spatial weights are multiplied element-wise with the reused and fused semantic features to output the spatial weighted features.

[0169] By fusing channel-weighted features with spatial-weighted features, semantic-channel-spatial-weighted refined features are generated.

[0170] Furthermore, it also includes a model training module, comprising:

[0171] The sample acquisition unit is used to acquire low-resolution images of defects.

[0172] The local training unit is used to train the initial edge focus feature extraction network of the initial feature focus refinement defect detection model locally using the defect-annotated low-resolution image. When the first loss function value converges, the intermediate feature focus refinement defect detection model is output.

[0173] The overall training unit is used to train the intermediate feature-focused and refined defect detection model based on the defect-annotated low-resolution image until the second loss function value converges, thus determining the target feature-focused and refined defect detection model.

[0174] Furthermore, the defect edge enhancement deep feature aggregation network includes an encoder and a decoder. The encoder includes an upsampling layer and a convolutional layer with activation functions, and the decoder includes a convolutional layer with activation functions and an image reconstruction layer; the local training unit includes:

[0175] The low-level extraction subunit is used in the initial edge focus feature extraction network of the initial feature focus refinement defect detection model. After the low-resolution image of defect annotation is processed by convolutional layer, convolutional recursive feature extraction is performed based on two cascaded convolutional recursive fusion layers to generate training low-level features.

[0176] The high-level extraction subunit is used to perform continuous feature processing on the training low-level features through two cascaded convolutional recursive fusion layers in the initial edge focusing feature extraction network to obtain the training high-level features.

[0177] The image reconstruction subunit is used to input the low-level and high-level training features into a pre-defined defect edge enhancement deep feature aggregation network for feature fusion enhancement and output a high-resolution training image.

[0178] The loss calculation subunit is used to calculate the first loss function value based on the training high-resolution image and the defect annotation low-resolution image corresponding to the defect annotation high-resolution image;

[0179] The model output sub-unit is used to iteratively optimize the model parameters in the initial edge focusing feature extraction network according to the first loss function value until the first loss function value converges, thus obtaining the intermediate feature focusing refinement defect detection model.

[0180] Furthermore, the image reconstruction subunit is specifically used for:

[0181] The first encoded feature is obtained by upsampling the low-level training features through an upsampling layer.

[0182] A convolutional layer with an activation function is used to perform feature mapping on the high-level training features to generate a second encoded feature;

[0183] The first and second encoded features are concatenated through channels to output the image encoded features;

[0184] The first decoding feature is obtained by continuously extracting image coding features through multiple convolutional layers with activation functions.

[0185] An image reconstruction layer is used to perform image super-resolution reconstruction based on the first decoding features to generate a training high-resolution image.

[0186] This invention also provides a computer device, including a memory and a processor, wherein the memory stores a computer program; when the computer program is executed by the processor, the processor performs the steps of the surface defect detection method based on feature focusing refinement as described in any of the above embodiments.

[0187] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the system and modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0188] In the embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between systems or units may be electrical, mechanical, or other forms.

[0189] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0190] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0191] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A surface defect detection method based on feature focusing and refinement, characterized in that, include: The image to be detected is acquired and input into a preset target feature focusing and refining defect detection model. The target feature focusing and refining defect detection model includes a target edge focusing feature extraction network, a target reuse and fusion semantic network, a target semantic-channel-spatial weighted refining network, and a target detection layer. The target edge focusing feature extraction network is used to extract multi-scale features from the image to be detected, and outputs multiple multi-scale focusing feature vectors of defect edges. The multi-scale focusing feature vectors of each defect edge are input into the target reuse and fusion semantic network for semantic enhancement, thereby generating multiple reuse and fusion semantic features. The target semantic-channel-spatial weighted refinement network is used to filter the features of each reused and fused semantic feature, thereby determining multiple semantic-channel-spatial weighted refinement features. The target detection layer performs defect localization and detection based on the semantic-channel-spatial weighted refinement features, and outputs the defect detection results of the image to be detected. The target edge focusing feature extraction network includes convolutional layers, convolutional recursive fusion layers, and spatial pyramid pooling layers. The convolutional recursive fusion layer includes a convolutional layer and a recursive convolutional enhancement layer. The recursive convolutional enhancement layer includes a recursive gated convolutional sub-layer, a feature segmentation sub-layer, and a recursive bottleneck block. The recursive bottleneck block includes multiple cascaded recursive gated convolutional sub-layers. The processing procedure of the convolutional recursive fusion layer includes: After performing convolution calculations on the convolutional input features input to the convolutional recursive fusion layer, recursive feature extraction is performed through a recursive gated convolutional sub-layer to generate intermediate convolutional features. The convolutional intermediate features are split using a feature segmentation sub-layer, and the first convolutional split feature and the second convolutional split feature are output. After the second convolution split feature is input into multiple cascaded recursive gated convolution sub-layers for continuous feature extraction, it is fused with the second convolution split feature to obtain the first convolution fusion feature. After extracting features from the first convolutional fusion feature by cascading multiple recursive gated convolutional sub-layers, the features are fused with the first convolutional fusion feature to determine the second convolutional fusion feature. After channel concatenation of the first convolution split feature, the first convolution fusion feature and the second convolution fusion feature, the convolution output feature is generated by inputting it into a recursive gated convolution sub-layer for convolution processing. The target reuse fusion semantic network includes an attention layer, a depthwise separable convolutional layer, and residual blocks; The target semantic-channel-spatial weighted refinement network includes a global average pooling layer, a global max pooling layer, a Conv_Silu layer, a Sigmoid activation function layer, and a Softmax activation function layer.

2. The surface defect detection method based on feature focusing and refinement according to claim 1, characterized in that, The step of extracting features from the image to be detected through the target edge focusing feature extraction network and outputting multiple defect edge multi-scale focusing feature vectors includes: After the image to be detected is input into the convolutional layer for convolution operation, convolutional recursive feature extraction is performed through two cascaded convolutional recursive fusion layers to obtain the first defect edge multi-scale focusing feature vector. A convolutional recursive fusion layer is used to process the first defect edge multi-scale focusing feature vector to generate the second defect edge multi-scale focusing feature vector. After performing feature mapping on the second defect edge multi-scale focusing feature vector through a convolutional recursive fusion layer, the input spatial pyramid pooling layer performs multi-scale pooling processing, and outputs the third defect edge multi-scale focusing feature vector.

3. The surface defect detection method based on feature focusing and refinement according to claim 1, characterized in that, The step involves inputting the multi-scale focused feature vectors of each defect edge into the target reuse and fusion semantic network for semantic enhancement, thereby generating multiple reuse and fusion semantic features, including: The multi-scale focusing feature vectors of each defect edge are respectively input into the cascaded attention layer and depth-separable convolutional layer for channel space feature extraction and fusion, corresponding to the first hemispherical perceptual embedding feature, the second hemispherical perceptual embedding feature and the third hemispherical perceptual embedding feature; After upsampling the third hemispherical perception embedding feature, it is fused with the second hemispherical perception embedding feature to generate the first semantic fusion feature; After upsampling the first semantic fusion feature, it is fused with the first hemispherical perception embedding feature to output the second semantic fusion feature. By extracting features from the first semantic fusion feature and the second semantic fusion feature using residual blocks, residual features and the first reused fusion semantic feature are obtained respectively. After downsampling the first multiplexed fusion semantic feature, it is fused with the residual feature, and then the feature is processed through the residual block to output the second multiplexed fusion semantic feature. After downsampling using the residual features, the features are fused with the third tomographic perception embedding features, and then feature extraction is performed via the residual blocks to output the third reused fusion semantic features.

4. The surface defect detection method based on feature focusing and refinement according to claim 1, characterized in that, The processing procedure of the target semantic-channel-spatial weighted refinement network includes: The reused and fused semantic features input to the target semantic-channel-space weighted refinement network are subjected to average pooling and max pooling respectively through a global average pooling layer and a global max pooling layer, generating average pooling features and max pooling features accordingly. After performing convolution operations on the average pooling features and the max pooling features through the Conv_Silu layer, feature fusion is performed to output channel features; The channel weights of the channel features are extracted using a Sigmoid activation function layer, and the channel weights are multiplied element-wise with the reused fusion semantic features to obtain channel-weighted features; The spatial weights of the reused and fused semantic features are extracted based on the Softmax activation function layer, and the spatial weights are multiplied element-wise with the reused and fused semantic features to output the spatial weighted features. The channel-weighted features and the spatial-weighted features are fused to generate semantic-channel-spatial-weighted refined features.

5. The surface defect detection method based on feature focusing and refinement according to claim 1, characterized in that, The training process of the target feature-focused and refined defect detection model includes: Obtain low-resolution images with defect annotations; The initial edge focusing feature extraction network of the initial feature focusing refinement defect detection model is trained locally using the defect-annotated low-resolution image. When the first loss function value converges, the intermediate feature focusing refinement defect detection model is output. The intermediate feature-focused refinement defect detection model is trained on the low-resolution image with the defect annotation until the second loss function value converges, thus determining the target feature-focused refinement defect detection model.

6. The surface defect detection method based on feature focusing and refinement according to claim 5, characterized in that, The process involves using the low-resolution image labeled with defects to train the initial edge focus feature extraction network of the initial feature-focused refinement defect detection model locally. When the first loss function converges, an intermediate feature-focused refinement defect detection model is output, including: In the initial edge focus feature extraction network of the initial feature focus refinement defect detection model, after the convolutional layer performs convolutional processing on the low-resolution image of the defect annotation, convolutional recursive feature extraction is performed based on two cascaded convolutional recursive fusion layers to generate training low-level features. The training low-level features are processed continuously by two convolutional recursive fusion layers cascaded in the initial edge focusing feature extraction network to obtain the training high-level features. The trained low-level features and the trained high-level features are input into a preset defect edge enhancement deep feature aggregation network for feature fusion enhancement, and the trained high-resolution image is output. The first loss function value is calculated based on the training high-resolution image and the defect-annotated high-resolution image corresponding to the defect-annotated low-resolution image; The model parameters in the initial edge focusing feature extraction network are iteratively optimized according to the first loss function value until the first loss function value converges, thus obtaining the intermediate feature focusing refinement defect detection model.

7. The surface defect detection method based on feature focusing and refinement according to claim 6, characterized in that, The defect edge enhancement deep feature aggregation network includes an encoder and a decoder. The encoder includes an upsampling layer and a convolutional layer with an activation function, and the decoder includes a convolutional layer with an activation function and an image reconstruction layer. The step of inputting the trained low-level features and the trained high-level features into the preset defect edge enhancement deep feature aggregation network for feature fusion enhancement, and outputting a training high-resolution image, includes: The training low-level features are upsampled by an upsampling layer to obtain the first encoded features; The training high-level features are mapped using a convolutional layer with an activation function to generate a second encoded feature; The first encoded feature and the second encoded feature are concatenated by channels to output the image encoded feature; The first decoded feature is obtained by continuously extracting features from the image coding features through multiple convolutional layers with activation functions. An image reconstruction layer is used to perform image super-resolution reconstruction based on the first decoding features to generate a training high-resolution image.

8. A surface defect detection system based on feature focusing refinement, characterized in that, include: The feature input module is used to acquire the image to be detected and input the image to be detected into a preset target feature focusing and refining defect detection model. The target feature focusing and refining defect detection model includes a target edge focusing feature extraction network, a target reuse and fusion semantic network, a target semantic-channel-spatial weighted refining network, and a target detection layer. The feature extraction module is used to perform multi-scale feature extraction on the image to be detected through the target edge focusing feature extraction network, and output multiple defect edge multi-scale focusing feature vectors; The feature enhancement module is used to input the multi-scale focused feature vectors of each defect edge into the target reuse and fusion semantic network for semantic enhancement, thereby generating multiple reuse and fusion semantic features. The feature refinement module is used to perform feature filtering on each of the multiplexed and fused semantic features using the target semantic-channel-space weighted refinement network, and to determine multiple semantic-channel-space weighted refinement features accordingly. The defect detection module is used to perform defect localization and detection based on the semantic-channel-spatial weighted refinement features of the target detection layer, and output the defect detection result of the image to be detected; The target edge focusing feature extraction network includes convolutional layers, convolutional recursive fusion layers, and spatial pyramid pooling layers. The convolutional recursive fusion layer includes a convolutional layer and a recursive convolutional enhancement layer. The recursive convolutional enhancement layer includes a recursive gated convolutional sub-layer, a feature segmentation sub-layer, and a recursive bottleneck block. The recursive bottleneck block includes multiple cascaded recursive gated convolutional sub-layers. The processing procedure of the convolutional recursive fusion layer includes: After performing convolution calculations on the convolutional input features input to the convolutional recursive fusion layer, recursive feature extraction is performed through a recursive gated convolutional sub-layer to generate intermediate convolutional features. The convolutional intermediate features are split using a feature segmentation sub-layer, and the first convolutional split feature and the second convolutional split feature are output. After the second convolution split feature is input into multiple cascaded recursive gated convolution sub-layers for continuous feature extraction, it is fused with the second convolution split feature to obtain the first convolution fusion feature. After extracting features from the first convolutional fusion feature by cascading multiple recursive gated convolutional sub-layers, the features are fused with the first convolutional fusion feature to determine the second convolutional fusion feature. After channel concatenation of the first convolution split feature, the first convolution fusion feature and the second convolution fusion feature, the convolution output feature is generated by inputting it into a recursive gated convolution sub-layer for convolution processing. The target reuse fusion semantic network includes an attention layer, a depthwise separable convolutional layer, and residual blocks; The target semantic-channel-spatial weighted refinement network includes a global average pooling layer, a global max pooling layer, a Conv_Silu layer, a Sigmoid activation function layer, and a Softmax activation function layer.

9. A computer device, characterized in that, The device includes a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor causes the processor to perform the steps of the surface defect detection method based on feature focusing refinement as described in any one of claims 1-7.

Images