Surface defect detection method and system based on feature focusing refinement

Abstract

The invention discloses a surface defect detection method and system based on feature focusing refinement, and relates to the technical field of defect detection. The method comprises the steps that a target feature focusing refining defect detection model comprises a target edge focusing feature extraction network, a target multiplexing fusion semantic network, a target semantic-channel-space weighted refining network and a target detection layer; outputting a plurality of defect edge multi-scale focusing feature vectors based on the to-be-detected image through a target edge focusing feature extraction network; inputting each defect edge multi-scale focusing feature vector into a target multiplexing fusion semantic network to correspondingly generate a plurality of multiplexing fusion semantic features; correspondingly determining a plurality of semantic-channel-space weighted refinement features based on each multiplexing fusion semantic feature by adopting a target semantic-channel-space weighted refinement network; and outputting a defect detection result of the to-be-detected image based on each semantic-channel-space weighted refinement feature through a target detection layer. And the small target defect detection precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of defect detection technology, and in particular to a surface defect detection method and system based on feature focusing and refinement. Background Art

[0002] With the in-depth integration of new-generation information technology and manufacturing, the industry is gradually shifting from quantity expansion to quality improvement. Especially in the clothing, luggage and other industries, the requirements for product surface quality are relatively high, and surface defect detection of products is an important part of quality control.

[0003] At present, existing surface defect detection methods usually adopt deep learning detection technology based on the YOLO model. By setting an attention mechanism in the main part of the YOLO model, the key channels and key spatial positions in the feature map are extracted and enhanced, and then the defects in the fused feature map are identified according to the feature information contained in the key channels and key spatial positions. However, after the attention mechanism is adopted, redundant features are not eliminated, resulting in low accuracy in small target defect detection. Summary of the invention

[0004] The present invention provides a surface defect detection method and system based on feature focusing and refinement, which solves the technical problem that the existing surface defect detection methods have low detection accuracy for small target defects.

[0005] A first aspect of the present invention provides a surface defect detection method based on feature focusing and refinement, comprising:

[0006] Acquire an image to be detected, and input the image to be detected into a preset target feature focusing and refinement defect detection model, wherein the target feature focusing and refinement defect detection model includes a target edge focusing feature extraction network, a target multiplexing fusion semantic network, a target semantic-channel-space weighted refinement network and a target detection layer;

[0007] Performing multi-scale feature extraction on the image to be inspected through the target edge focusing feature extraction network, and outputting a plurality of defect edge multi-scale focusing feature vectors;

[0008] Inputting each of the defect edge multi-scale focused feature vectors into the target multiplexing fusion semantic network for semantic enhancement, and correspondingly generating a plurality of multiplexing fusion semantic features;

[0009] The target semantic-channel-space weighted refinement network is used to perform feature filtering on each of the multiplexed fusion semantic features, and a plurality of semantic-channel-space weighted refinement features are correspondingly determined;

[0010] Defect location detection is performed through the target detection layer based on each of the semantic-channel-space weighted refinement features, and a defect detection result of the image to be detected is output.

[0011] Optionally, the target edge focusing feature extraction network includes a convolution layer, a convolution recursive fusion layer, and a spatial pyramid pooling layer; the feature extraction of the image to be detected by the target edge focusing feature extraction network and output of multiple defect edge multi-scale focusing feature vectors include:

[0012] After the image to be detected is input into the convolution layer for convolution operation, convolution recursive feature extraction is performed through two cascaded convolution recursive fusion layers to obtain a first defect edge multi-scale focused feature vector;

[0013] Using a convolutional recursive fusion layer to perform feature processing on the first defect edge multi-scale focused feature vector to generate a second defect edge multi-scale focused feature vector;

[0014] After feature mapping is performed on the second defect edge multi-scale focused feature vector through the convolution recursive fusion layer, the feature vector is input into the spatial pyramid pooling layer for multi-scale pooling processing, and a third defect edge multi-scale focused feature vector is output.

[0015] Optionally, the convolutional recursive fusion layer includes a convolutional layer and a recursive convolutional enhancement layer, the recursive convolutional enhancement layer includes a recursive gated convolutional sublayer, a feature segmentation sublayer and a recursive bottleneck block, and the recursive bottleneck block includes a plurality of cascaded recursive gated convolutional sublayers; the processing process of the convolutional recursive fusion layer includes:

[0016] After the convolution input features input to the convolution recursive fusion layer are convolved by a convolution layer, recursive feature extraction is performed by a recursive gated convolution sublayer to generate convolution intermediate features;

[0017] Using a feature segmentation sublayer to perform feature splitting on the convolution intermediate feature, and outputting a first convolution split feature and a second convolution split feature;

[0018] After inputting the second convolution split feature into a plurality of cascaded recursive gated convolution sub-layers for continuous feature extraction, the feature is fused with the second convolution split feature to obtain a first convolution fusion feature;

[0019] After extracting features step by step on the first convolutional fusion feature based on a plurality of cascaded recursive gated convolutional sub-layers, the first convolutional fusion feature is subjected to feature fusion to determine a second convolutional fusion feature;

[0020] After channel splicing of the first convolution split feature, the first convolution fusion feature and the second convolution fusion feature, they are input into the recursive gated convolution sublayer for convolution processing to generate convolution output features.

[0021] Optionally, the target reused fusion semantic network includes an attention layer, a depth-separable convolutional layer and a residual block; the multi-scale focused feature vectors of the defect edges are input into the target reused fusion semantic network for semantic enhancement, and a plurality of reused fusion semantic features are correspondingly generated, including:

[0022] Input the multi-scale focused feature vectors of the defect edges into the cascaded attention layer and the depth-separable convolution layer respectively for channel space feature extraction and fusion, and obtain the first tomography-aware embedding feature, the second tomography-aware embedding feature and the third tomography-aware embedding feature accordingly;

[0023] After upsampling the third tomographic-aware embedding feature, the feature fusion is performed with the second tomographic-aware embedding feature to generate a first semantic fusion feature;

[0024] After upsampling the first semantic fusion feature, the feature fusion is performed with the first tomography-aware embedding feature to output a second semantic fusion feature;

[0025] Performing feature extraction on the first semantic fusion feature and the second semantic fusion feature respectively through a residual block, and obtaining a residual feature and a first multiplexed fusion semantic feature respectively;

[0026] After downsampling the first multiplexed fused semantic feature, perform feature fusion with the residual feature, perform feature processing through a residual block, and output a second multiplexed fused semantic feature;

[0027] The residual feature is down-sampled and then fused with the third tomography-aware embedding feature, and feature extraction is performed via a residual block to output a third multiplexed fused semantic feature.

[0028] Optionally, the target semantic-channel-space weighted refinement network includes a global average pooling layer, a global maximum pooling layer, a Conv_Silu layer, a Sigmoid activation function layer and a Softmax activation function layer; the processing process of the target semantic-channel-space weighted refinement network includes:

[0029] The reused fusion semantic features input into the target semantic-channel-space weighted refinement network are respectively average pooled and maximum pooled through a global average pooling layer and a global maximum pooling layer, and average pooling features and maximum pooling features are generated accordingly;

[0030] The average pooling feature and the maximum pooling feature are respectively convolved by the Conv_Silu layer, and then feature fusion is performed to output channel features;

[0031] A Sigmoid activation function layer is used to extract the channel weight of the channel feature, and the channel weight is multiplied element by element by the multiplexed fusion semantic feature to obtain a channel weighted feature;

[0032] Extracting the spatial weight of the multiplexed fusion semantic feature based on the Softmax activation function layer, and multiplying the spatial weight by the multiplexed fusion semantic feature element by element, and outputting the spatial weighted feature;

[0033] The channel weighted features are fused with the space weighted features to generate semantic-channel-space weighted refined features.

[0034] Optionally, the training process of the target feature focused refinement defect detection model includes:

[0035] Obtain defect-annotated low-resolution images;

[0036] The defect-annotated low-resolution image is used to perform local model training on an initial edge-focused feature extraction network of an initial feature-focused and refined defect detection model, and when the first loss function value converges, an intermediate feature-focused and refined defect detection model is output;

[0037] Based on the defect-annotated low-resolution image, the intermediate feature focusing and refinement defect detection model is trained as a whole until the second loss function value converges, and a target feature focusing and refinement defect detection model is determined.

[0038] Optionally, the defect-annotated low-resolution image is used to perform local model training on an initial edge-focused feature extraction network of an initial feature-focused and refined defect detection model, and when the first loss function converges, an intermediate feature-focused and refined defect detection model is output, including:

[0039] In the initial edge focus feature extraction network of the initial feature focus refinement defect detection model, after convolution processing is performed on the defect annotated low-resolution image using a convolution layer, convolution recursive feature extraction is performed based on two cascaded convolution recursive fusion layers to generate training low-level features;

[0040] Performing continuous feature processing on the training low-level features through two convolutional recursive fusion layers cascaded in the initial edge-focused feature extraction network to obtain training high-level features;

[0041] Input the training low-level features and the training high-level features into a preset defect edge enhancement deep feature aggregation network for feature fusion enhancement, and output a training high-resolution image;

[0042] Calculating a first loss function value based on the training high-resolution image and the defect-annotated high-resolution image corresponding to the defect-annotated low-resolution image;

[0043] The model parameters in the initial edge focusing feature extraction network are iteratively optimized according to the first loss function value until the first loss function value converges to obtain an intermediate feature focusing refinement defect detection model.

[0044] Optionally, the defect edge enhancement deep feature aggregation network includes an encoder and a decoder, the encoder includes an upsampling layer and a convolution layer with an activation function, and the decoder includes a convolution layer with an activation function and an image reconstruction layer; the step of inputting the training low-level features and the training high-level features into a preset defect edge enhancement deep feature aggregation network for feature fusion enhancement and outputting a training high-resolution image includes:

[0045] Upsampling the training low-level features through an upsampling layer to obtain a first encoding feature;

[0046] Using a convolutional layer with an activation function to perform feature mapping on the training high-level features to generate a second encoding feature;

[0047] Perform channel concatenation of the first coding feature and the second coding feature, and output an image coding feature;

[0048] Continuously extracting features of the image coding features through multiple convolutional layers with activation functions to obtain first decoding features;

[0049] An image reconstruction layer is used to perform image super-resolution reconstruction based on the first decoded features to generate a training high-resolution image.

[0050] A second aspect of the present invention provides a surface defect detection system based on feature focusing and refinement, comprising:

[0051] A feature input module is used to obtain an image to be detected and input the image to be detected into a preset target feature focusing and refinement defect detection model, wherein the target feature focusing and refinement defect detection model includes a target edge focusing feature extraction network, a target multiplexing fusion semantic network, a target semantic-channel-space weighted refinement network and a target detection layer;

[0052] A feature extraction module, used to perform multi-scale feature extraction on the image to be detected through the target edge focusing feature extraction network, and output a plurality of defect edge multi-scale focusing feature vectors;

[0053] A feature enhancement module, used for inputting the multi-scale focused feature vectors of each defect edge into the target multiplexing fusion semantic network for semantic enhancement, and correspondingly generating a plurality of multiplexing fusion semantic features;

[0054] A feature refinement module, used to use the target semantic-channel-space weighted refinement network to perform feature filtering on each of the multiplexed fusion semantic features, and correspondingly determine a plurality of semantic-channel-space weighted refinement features;

[0055] The defect detection module is used to perform defect location detection based on each of the semantic-channel-space weighted refinement features through the target detection layer, and output the defect detection result of the image to be detected.

[0056] A third aspect of the present invention provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the surface defect detection method based on feature focus refinement as described in any one of the above items.

[0057] It can be seen from the above technical solutions that the present invention has the following advantages:

[0058] The above scheme of the present invention, after inputting the image to be detected into the target feature focusing and refinement defect detection model, firstly focuses on the small target features through the target edge focusing feature extraction network to perform multi-scale feature extraction on the image to be detected, and then better captures the effective information about the defect features based on the target reuse fusion semantic network, and then uses the target semantic-channel-space weighted refinement network to filter redundant features, and finally outputs to the target detection layer to locate the defect area, thereby improving the accuracy of small target defect detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0060] Figure 1 A flow chart of the steps of a surface defect detection method based on feature focusing and refinement provided by an embodiment of the present invention;

[0061] Figure 2 A framework diagram of a feature-focused and refined defect detection model provided by an embodiment of the present invention;

[0062] Figure 3 A schematic diagram of the structure of a recursive convolution enhancement layer provided in an embodiment of the present invention;

[0063] Figure 4 A schematic diagram of the structure of a defect edge enhancement deep feature aggregation network provided in an embodiment of the present invention;

[0064] Figure 5 A schematic diagram of the structure of a multiplexed fusion semantic network provided by an embodiment of the present invention;

[0065] Figure 6 A schematic diagram of the structure of a semantic-channel-space weighted refinement network provided in an embodiment of the present invention;

[0066] Figure 7 A structural block diagram of a surface defect detection system based on feature focus refinement provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The embodiments of the present invention provide a surface defect detection method and system based on feature focusing and refinement, which are used to solve the technical problem that the existing surface defect detection methods have low detection accuracy for small target defects.

[0068] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0069] See also Figure 1 , Figure 1 A flow chart of the steps of a surface defect detection method based on feature focusing and refinement provided in an embodiment of the present invention.

[0070] The present invention provides a surface defect detection method based on feature focusing and refinement, comprising:

[0071] Step 101, obtain the image to be detected, and input the image to be detected into a preset target feature focusing and refinement defect detection model, wherein the target feature focusing and refinement defect detection model includes a target edge focusing feature extraction network, a target multiplexing fusion semantic network, a target semantic-channel-space weighted refinement network and a target detection layer.

[0072] It should be noted that the image to be inspected refers to the product image to be inspected for surface defects. The target feature focusing and refinement defect detection model provided in this embodiment is improved based on the existing YOLOv8. Figure 2 As shown, it includes the target edge focusing feature extraction network, the target reuse fusion semantic network (RBANet), the target semantic-channel-space weighted refinement network (PFF) and the target detection layer (Detect).

[0073] Step 102: Perform multi-scale feature extraction on the image to be inspected through a target edge focusing feature extraction network, and output multiple defect edge multi-scale focusing feature vectors.

[0074] like Figure 2 As shown, the target edge focusing feature extraction network includes a convolution layer (Conv), a convolution recursive fusion layer and a spatial pyramid pooling layer (SPPF); step 102 includes the following sub-steps:

[0075] After the image to be detected is input into the convolution layer for convolution operation, convolution recursive feature extraction is performed through two cascaded convolution recursive fusion layers to obtain the multi-scale focused feature vector of the first defect edge;

[0076] A convolutional recursive fusion layer is used to perform feature processing on the first defect edge multi-scale focused feature vector to generate a second defect edge multi-scale focused feature vector;

[0077] After feature mapping of the second defect edge multi-scale focused feature vector is performed through the convolutional recursive fusion layer, it is input into the spatial pyramid pooling layer for multi-scale pooling processing, and the third defect edge multi-scale focused feature vector is output.

[0078] Furthermore, if Figure 2 and Figure 3 As shown in the figure, the convolutional recursive fusion layer includes a convolutional layer (Conv) and a recursive convolutional enhancement layer (GnC2f). The recursive convolutional enhancement layer includes a recursive gated convolutional sublayer (GnConv), a feature segmentation sublayer (split) and a recursive bottleneck block (GnDarknetBottleneck). The recursive bottleneck block includes multiple cascaded recursive gated convolutional sublayers. The processing process of the convolutional recursive fusion layer includes:

[0079] After the convolutional input features of the input convolutional recursive fusion layer are convolved by the convolutional layer, recursive feature extraction is performed through the recursive gated convolutional sublayer to generate convolutional intermediate features;

[0080] The feature segmentation sublayer is used to perform feature splitting on the convolution intermediate features, and the first convolution split features and the second convolution split features are output;

[0081] After inputting the second convolution split feature into multiple cascaded recursive gated convolution sub-layers for continuous feature extraction, the feature is fused with the second convolution split feature to obtain the first convolution fusion feature;

[0082] After extracting features of the first convolution fusion feature step by step based on multiple cascaded recursive gated convolution sub-layers, the first convolution fusion feature is fused with the first convolution fusion feature to determine the second convolution fusion feature;

[0083] After channel splicing of the first convolution split feature, the first convolution fusion feature, and the second convolution fusion feature, they are input into the recursive gated convolution sublayer for convolution processing to generate convolution output features.

[0084] It should be noted that the target edge focusing feature extraction network is based on CSPDarkNet-53 and introduces a recursive gated convolution sublayer (GnConv) into the C2f module. The GnC2f structure can extend the second-order interaction in self-attention to any order, reduce the amount of calculation, and thus better identify defective product images with small area and low contrast. It can also more efficiently integrate local features and global information, enhancing the network's ability to express contextual associations.

[0085] Step 103: Input the multi-scale focused feature vectors of each defect edge into the target multiplexing fusion semantic network for semantic enhancement, and generate a plurality of multiplexing fusion semantic features accordingly.

[0086] like Figure 5 As shown, the target reuse fusion semantic network (RBANet) includes an attention layer (CBAM), a depth-separable convolutional layer (DSConv) and a residual block (ResBlock); step 103 includes the following sub-steps:

[0087] S11. Input the multi-scale focused feature vectors of each defect edge into the cascaded attention layer and the depth-separable convolution layer for channel space feature extraction and fusion, and obtain the first layer tomography-aware embedded features, the second layer tomography-aware embedded features and the third layer tomography-aware embedded features accordingly.

[0088] It should be noted that each defect edge multiscale focused feature vector, such as the first defect edge multiscale focused feature vector, the second defect edge multiscale focused feature vector and the third defect edge multiscale focused feature vector, are defect edge multiscale focused feature vectors at different scales. First, each defect edge multiscale focused feature vector is used as the attention input feature of the attention layer, and the information is extracted in the channel dimension and the spatial dimension in turn through the attention layer. Placing the attention layer after the target edge focused feature extraction network can better capture the effective information about the defect features, and improve the utilization rate of the defect position by re-adjusting the channel features and spatial features. The specific process of the attention layer is determined by the following formula:

[0089] ;

[0090] ;

[0091] In the formula, To pay attention to the input features, To handle the channel dimension, is the global average pooling operation, is the maximum pooling operation, is a multi-layer perceptron, is the sigmoid activation function, To handle the spatial dimension, It is a 7*7 convolution. is the feature after processing by channel dimension;

[0092] Then, the attention output features of the attention layer are passed through the depthwise separable convolutional layer (DSConv) to further fuse the features while reducing the convolution parameters.

[0093] S12: After upsampling the third layer-aware embedding feature, perform feature fusion with the second layer-aware embedding feature to generate a first semantic fusion feature.

[0094] S13, after upsampling the first semantic fusion feature, perform feature fusion with the first tomography-aware embedding feature to output a second semantic fusion feature.

[0095] S14, extracting features of the first semantic fusion feature and the second semantic fusion feature respectively through the residual block, and obtaining a residual feature and a first multiplexed fusion semantic feature correspondingly.

[0096] S15, down-sampling the first multiplexed fused semantic feature, performing feature fusion with the residual feature, performing feature processing through the residual block, and outputting a second multiplexed fused semantic feature.

[0097] S16, after downsampling the residual feature, perform feature fusion with the third tomography-aware embedding feature, perform feature extraction through the residual block, and output the third multiplexed fusion semantic feature.

[0098] It should be noted that the first tomography-aware embedding features, the second tomography-aware embedding features and the third tomography-aware embedding features output by the depthwise separable convolutional layer are subjected to multi-scale feature fusion through an upsampling fusion layer based on the FPN structure, and the first semantic fusion features and the second semantic fusion features output after fusion are subjected to residual processing through a residual block structure based on DSConv, and then processed through a downsampling fusion layer based on the PAN structure, and finally, the hierarchical semantically enhanced reused fusion semantic features are obtained through a residual block structure based on DSConv.

[0099] Step 104: Use the target semantic-channel-space weighted refinement network to perform feature filtering on each multiplexed fusion semantic feature, and correspondingly determine a plurality of semantic-channel-space weighted refinement features.

[0100] like Figure 6As shown in the figure, the target semantic-channel-space weighted refinement network (PFF) includes a global average pooling layer (Adaptive average pooling), a global maximum pooling layer (Adaptive maximum pooling), a Conv_Silu layer, a Sigmoid activation function layer, and a Softmax activation function layer; the processing process of the target semantic-channel-space weighted refinement network includes:

[0101] The reused fusion semantic features of the input target semantic-channel-space weighted refinement network are respectively average pooled and maximum pooled through the global average pooling layer and the global maximum pooling layer to generate average pooling features and maximum pooling features respectively;

[0102] Through the Conv_Silu layer, the average pooling features and the maximum pooling features are convolved and then feature fused to output channel features;

[0103] The Sigmoid activation function layer is used to extract the channel weight of the channel feature, and the channel weight is multiplied element by element with the reused fusion semantic feature to obtain the channel weighted feature;

[0104] The spatial weight of the reused fusion semantic feature is extracted based on the Softmax activation function layer, and the spatial weight is multiplied by the reused fusion semantic feature element by element to output the spatial weighted feature;

[0105] The channel-weighted features are fused with the spatial-weighted features to generate semantic-channel-space weighted refined features.

[0106] It should be noted that the first multiplexed fusion semantic feature, the second multiplexed fusion semantic feature and the third multiplexed fusion semantic feature are sequentially input into the target semantic-channel-space weighted refinement network for processing. First, the channel adaptive weight is calculated by channel refinement guided by the global average pooling layer and the global maximum pooling layer, and the spatial weight is determined by the Softmax activation function layer, and then multiplied with the multiplexed fusion semantic feature to filter out interference features. The specific process includes:

[0107] ;

[0108] ;

[0109] ;

[0110] In the formula, is the channel weight, is the sigmoid activation function, To reuse the fusion semantic features, It is a global average pooling operation followed by convolution and SiLu activation function. After the maximum pooling operation, it passes through the convolution and SiLu activation function. is the spatial weight, is the Softmax activation function, Refine features for semantic-channel-spatial weighting.

[0111] Step 105: Perform defect location detection based on the semantic-channel-space weighted refinement features through the target detection layer, and output the defect detection result of the image to be detected.

[0112] It should be noted that in the defect detection results, the defects are located and marked through the edge focus prediction box, and the target detection layer uses the original Detect detection layer of YOLOv8 for detection. The specific application process can refer to the existing technology and will not be repeated here.

[0113] Furthermore, the target feature focused refinement defect detection model provided in this embodiment is determined by staged training after the initial feature focused refinement defect detection model is obtained by improving the existing YOLOv8. The initial feature focused refinement defect detection model includes an initial edge focused feature extraction network, an initial multiplexed fusion semantic network (RBANet), an initial semantic-channel-space weighted refinement network (PFF) and an initial detection layer (Detect). In specific implementation, the training process of the target feature focused refinement defect detection model includes:

[0114] S31, obtaining a low-resolution image with defect annotations.

[0115] It should be noted that the defect-annotated low-resolution image refers to an image in which defects have been annotated and is of low resolution. In a specific implementation, the defect annotation can be reflected in the use of a rectangular frame in the image to mark the coordinates and length and width of the defect.

[0116] Furthermore, before step S31, a defect annotation training image can be obtained first, and a defect annotation low-resolution image for input model training can be obtained by preprocessing the defect annotation low-resolution training image, wherein the preprocessing can include operations such as image cropping, noise filtering, scale normalization and contrast adjustment.

[0117] S32. Use the defect-annotated low-resolution image to perform local model training on the initial edge-focused feature extraction network of the initial feature-focused refinement defect detection model, and when the first loss function value converges, output the intermediate feature-focused refinement defect detection model.

[0118] like Figure 2 As shown in FIG. 1 , the determination process of the intermediate feature focused refinement defect detection model includes:

[0119] S321. In the initial edge focusing feature extraction network of the initial feature focusing refinement defect detection model, after the defect annotation low-resolution image is convolved by a convolution layer, convolution recursive feature extraction is performed based on two cascaded convolution recursive fusion layers to generate a training lower level feature (Lower Lever Feature).

[0120] S322, performing continuous feature processing on the training low-level features through two convolutional recursive fusion layers cascaded in the initial edge-focused feature extraction network to obtain the training high-level features (High Lever Feature).

[0121] S323, input the training low-level features and the training high-level features into a preset defect edge enhancement deep feature aggregation network for feature fusion enhancement, and output a training high-resolution image.

[0122] Furthermore, if Figure 4 As shown, the defect edge enhancement deep feature aggregation network includes an encoder (Encoder) and a decoder (Decoder), the encoder includes an upsampling layer (Upsample) and a convolution layer (CR) with an activation function, the decoder includes a convolution layer (CR) with an activation function and an image reconstruction layer (EDSR), and in a specific implementation, the convolution layer (CR) with an activation function includes a 3×3 convolution and a ReLu activation function; the process of S323 includes:

[0123] Up-sampling the training low-level features through the up-sampling layer to obtain the first encoding features;

[0124] Use a convolutional layer with an activation function to perform feature mapping on the training high-level features to generate the second encoding features;

[0125] Channel-joining the first coding feature and the second coding feature to output an image coding feature;

[0126] Continuously extracting the image coding features through multiple layers of convolutional layers with activation functions to obtain the first decoding features;

[0127] An image reconstruction layer is used to perform image super-resolution reconstruction based on the first decoded features to generate a training high-resolution image.

[0128] It should be noted that in the image reconstruction layer (EDSR), the first decoded feature is residually processed through 32 residual blocks to generate a second decoded feature vector. After the second decoded feature vector is feature reorganized through 3×3 convolution, it is upsampled based on the sub-pixel convolution layer to obtain a training high-resolution image that is twice the size of the defect annotation low-resolution image.

[0129] S324, calculating a first loss function value based on the defect-annotated high-resolution image corresponding to the training high-resolution image and the defect-annotated low-resolution image.

[0130] It should be noted that the L1 loss calculation is performed on the high-resolution training image output by the defect edge enhancement deep feature aggregation network and the defect annotated high-resolution image corresponding to the defect annotated low-resolution image. The calculation process includes:

[0131] ;

[0132] In the formula, is the first loss function value, is the convolution weight from layer 1 to layer L, is the bias from layer 1 to layer L, is the height of the low-resolution image, is the width of the low-resolution image, is the magnification, High-resolution images for defect annotation The pixel value at Annotate low-resolution images for defects The pixel value at For training high-resolution images The pixel value at .

[0133] S325. Iteratively optimize the model parameters in the initial edge focusing feature extraction network according to the first loss function value until the first loss function value converges to obtain an intermediate feature focusing refinement defect detection model.

[0134] It should be noted that when the first stage model training is performed on the initial feature focused refinement defect detection model, the model parameters in the initial edge focused feature extraction network are iteratively updated by back propagation according to the first loss function value until the first loss function value converges, thereby obtaining the intermediate feature focused refinement defect detection model. The feature extraction capability of convolution downsampling in the initial edge focused feature extraction network is optimized through pre-training, focusing on extracting small target features. Since the spatial pyramid pooling layer (SPPF) is mainly for preliminary feature fusion of the last layer of features, it does not involve downsampling feature extraction and does not participate in the first stage training.

[0135] S33. Based on the defect-annotated low-resolution image, the intermediate feature focusing and refinement defect detection model is trained as a whole until the second loss function value converges, and the target feature focusing and refinement defect detection model is determined.

[0136] It should be noted that, during the second-stage model training, the defect-annotated low-resolution image is used to train the intermediate feature focus refinement defect detection model as a whole, and finally the second loss function value is calculated based on the output of the intermediate feature focus refinement defect detection model, and the model parameters in the intermediate feature focus refinement defect detection model are iteratively optimized according to the second loss function value until the second loss function value converges to determine the target feature focus refinement defect detection model; in specific implementation, the second loss function value can adopt the quality focus loss function Calculate to obtain the edge focus prediction frame, the quality focus loss function specifically includes the following formula:

[0137] ;

[0138] In the formula, is the score of the label multiplied by the corresponding IOU, is the classification score after sigmoid operation, It is an adjustable parameter.

[0139] In an embodiment of the present invention, after the image to be detected is input into the target feature focusing and refinement defect detection model, the small target features are first focused on by the target edge focusing feature extraction network to perform multi-scale feature extraction on the image to be detected, and then the effective information about the defect features is better captured based on the target reuse fusion semantic network, and then the target semantic-channel-space weighted refinement network is used to filter the redundant features, and finally the features are output to the target detection layer to locate the defect area, thereby improving the accuracy of small target defect detection.

[0140] See also Figure 7 , Figure 7 A structural block diagram of a surface defect detection system based on feature focus refinement provided in an embodiment of the present invention.

[0141] The present invention provides a surface defect detection system based on feature focusing and refinement, comprising:

[0142] The feature input module 701 is used to obtain the image to be detected and input the image to be detected into a preset target feature focusing and refinement defect detection model, wherein the target feature focusing and refinement defect detection model includes a target edge focusing feature extraction network, a target multiplexing fusion semantic network, a target semantic-channel-space weighted refinement network and a target detection layer;

[0143] A feature extraction module 702 is used to perform multi-scale feature extraction on the image to be inspected through a target edge focusing feature extraction network, and output a plurality of defect edge multi-scale focusing feature vectors;

[0144] The feature enhancement module 703 is used to input the multi-scale focused feature vectors of each defect edge into the target multiplexing fusion semantic network for semantic enhancement, and correspondingly generate multiple multiplexing fusion semantic features;

[0145] The feature refinement module 704 is used to perform feature filtering on each multiplexed fusion semantic feature using a target semantic-channel-space weighted refinement network, and correspondingly determine a plurality of semantic-channel-space weighted refinement features;

[0146] The defect detection module 705 is used to perform defect location detection based on each semantic-channel-space weighted refinement feature through the target detection layer, and output the defect detection result of the image to be detected.

[0147] Furthermore, the target edge focusing feature extraction network includes a convolution layer, a convolution recursive fusion layer and a spatial pyramid pooling layer; the feature extraction module 702 is specifically used for:

[0148] After the image to be detected is input into the convolution layer for convolution operation, convolution recursive feature extraction is performed through two cascaded convolution recursive fusion layers to obtain the multi-scale focused feature vector of the first defect edge;

[0149] A convolutional recursive fusion layer is used to perform feature processing on the first defect edge multi-scale focused feature vector to generate a second defect edge multi-scale focused feature vector;

[0150] After feature mapping of the second defect edge multi-scale focused feature vector is performed through the convolutional recursive fusion layer, it is input into the spatial pyramid pooling layer for multi-scale pooling processing, and the third defect edge multi-scale focused feature vector is output.

[0151] Furthermore, the convolution-recursive fusion layer includes a convolution layer and a recursive convolution enhancement layer, the recursive convolution enhancement layer includes a recursive gated convolution sublayer, a feature segmentation sublayer and a recursive bottleneck block, and the recursive bottleneck block includes a plurality of cascaded recursive gated convolution sublayers; the processing process of the convolution-recursive fusion layer includes:

[0152] After the convolutional input features of the input convolutional recursive fusion layer are convolved by the convolutional layer, recursive feature extraction is performed through the recursive gated convolutional sublayer to generate convolutional intermediate features;

[0153] The feature segmentation sublayer is used to perform feature splitting on the convolution intermediate features, and the first convolution split features and the second convolution split features are output;

[0154] After inputting the second convolution split feature into multiple cascaded recursive gated convolution sub-layers for continuous feature extraction, the feature is fused with the second convolution split feature to obtain the first convolution fusion feature;

[0155] After extracting features of the first convolution fusion feature step by step based on multiple cascaded recursive gated convolution sub-layers, the first convolution fusion feature is fused with the first convolution fusion feature to determine the second convolution fusion feature;

[0156] After channel splicing of the first convolution split feature, the first convolution fusion feature, and the second convolution fusion feature, they are input into the recursive gated convolution sublayer for convolution processing to generate convolution output features.

[0157] Furthermore, the target reuse fusion semantic network includes an attention layer, a depth-separable convolutional layer and a residual block; the feature enhancement module 703 is specifically used for:

[0158] The multi-scale focused feature vectors of each defect edge are respectively input into the cascaded attention layer and the depth-separable convolution layer for channel space feature extraction and fusion, and the first layer tomography-aware embedded features, the second layer tomography-aware embedded features and the third layer tomography-aware embedded features are obtained accordingly;

[0159] After upsampling the third layer tomographic-aware embedded feature, feature fusion is performed with the second layer tomographic-aware embedded feature to generate a first semantic fusion feature;

[0160] After upsampling the first semantic fusion feature, the feature fusion is performed with the first tomography-aware embedding feature to output a second semantic fusion feature;

[0161] The first semantic fusion feature and the second semantic fusion feature are respectively extracted by the residual block, and the residual feature and the first multiplexed fusion semantic feature are correspondingly obtained;

[0162] After downsampling the first multiplexed fused semantic feature, the feature is fused with the residual feature, and the feature is processed through the residual block to output a second multiplexed fused semantic feature;

[0163] The residual feature is downsampled and then fused with the third tomography-aware embedding feature, and feature extraction is performed through the residual block to output the third multiplexed fused semantic feature.

[0164] Furthermore, the target semantic-channel-space weighted refinement network includes a global average pooling layer, a global maximum pooling layer, a Conv_Silu layer, a Sigmoid activation function layer, and a Softmax activation function layer; the processing process of the target semantic-channel-space weighted refinement network includes:

[0165] The reused fusion semantic features of the input target semantic-channel-space weighted refinement network are respectively average pooled and maximum pooled through the global average pooling layer and the global maximum pooling layer to generate average pooling features and maximum pooling features respectively;

[0166] Through the Conv_Silu layer, the average pooling features and the maximum pooling features are convolved and then feature fused to output channel features;

[0167] The Sigmoid activation function layer is used to extract the channel weight of the channel feature, and the channel weight is multiplied element by element with the reused fusion semantic feature to obtain the channel weighted feature;

[0168] The spatial weight of the reused fusion semantic feature is extracted based on the Softmax activation function layer, and the spatial weight is multiplied by the reused fusion semantic feature element by element to output the spatial weighted feature;

[0169] The channel-weighted features are fused with the spatial-weighted features to generate semantic-channel-space weighted refined features.

[0170] Furthermore, a model training module is also included, including:

[0171] A sample acquisition unit, used to acquire a low-resolution image with defect annotations;

[0172] A local training unit is used to perform local model training on an initial edge focus feature extraction network of an initial feature focus refinement defect detection model using a defect annotated low-resolution image, and output an intermediate feature focus refinement defect detection model when a first loss function value converges;

[0173] The overall training unit is used to perform overall model training on the intermediate feature focusing and refinement defect detection model based on the defect annotated low-resolution image until the second loss function value converges, and determine the target feature focusing and refinement defect detection model.

[0174] Furthermore, the defect edge enhancement deep feature aggregation network includes an encoder and a decoder, the encoder includes an upsampling layer and a convolution layer with an activation function, and the decoder includes a convolution layer with an activation function and an image reconstruction layer; a local training unit includes:

[0175] A low-level extraction subunit is used to perform convolution processing on the defect annotation low-resolution image using a convolution layer in the initial edge-focused feature extraction network of the initial feature-focused refinement defect detection model, and then perform convolution recursive feature extraction based on two cascaded convolution recursive fusion layers to generate training low-level features;

[0176] A high-level extraction subunit is used to perform continuous feature processing on the training low-level features through two convolutional recursive fusion layers cascaded in the initial edge-focused feature extraction network to obtain training high-level features;

[0177] An image reconstruction subunit is used to input the training low-level features and the training high-level features into a preset defect edge enhancement deep feature aggregation network for feature fusion enhancement, and output a training high-resolution image;

[0178] A loss value calculation subunit, used to calculate a first loss function value based on the defect annotated high-resolution image corresponding to the training high-resolution image and the defect annotated low-resolution image;

[0179] The model output subunit is used to iteratively optimize the model parameters in the initial edge focusing feature extraction network according to the first loss function value until the first loss function value converges to obtain an intermediate feature focusing refinement defect detection model.

[0180] Furthermore, the image reconstruction subunit is specifically used for:

[0181] Up-sampling the training low-level features through the up-sampling layer to obtain the first encoding features;

[0182] Use a convolutional layer with an activation function to perform feature mapping on the training high-level features to generate the second encoding features;

[0183] Channel-joining the first coding feature and the second coding feature to output an image coding feature;

[0184] Continuously extracting the image coding features through multiple layers of convolutional layers with activation functions to obtain the first decoding features;

[0185] An image reconstruction layer is used to perform image super-resolution reconstruction based on the first decoded features to generate a training high-resolution image.

[0186] An embodiment of the present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory; when the computer program is executed by the processor, the processor executes the steps of the surface defect detection method based on feature focusing and refinement as in any of the above embodiments.

[0187] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described system and module can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0188] In the several embodiments provided in the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0189] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0190] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0191] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A surface defect detection method based on feature focusing and refinement, characterized in that: include: Acquire an image to be detected, and input the image to be detected into a preset target feature focusing and refinement defect detection model, wherein the target feature focusing and refinement defect detection model includes a target edge focusing feature extraction network, a target multiplexing fusion semantic network, a target semantic-channel-space weighted refinement network and a target detection layer; Performing multi-scale feature extraction on the image to be inspected through the target edge focusing feature extraction network, and outputting a plurality of defect edge multi-scale focusing feature vectors; Inputting each of the defect edge multi-scale focused feature vectors into the target multiplexing fusion semantic network for semantic enhancement, and correspondingly generating a plurality of multiplexing fusion semantic features; The target semantic-channel-space weighted refinement network is used to perform feature filtering on each of the multiplexed fusion semantic features, and a plurality of semantic-channel-space weighted refinement features are correspondingly determined; Defect location detection is performed through the target detection layer based on each of the semantic-channel-space weighted refinement features, and a defect detection result of the image to be detected is output.

2. The surface defect detection method based on feature focusing and refinement according to claim 1 is characterized in that: The target edge focusing feature extraction network includes a convolution layer, a convolution recursive fusion layer and a spatial pyramid pooling layer; the target edge focusing feature extraction network is used to extract features from the image to be detected, and multiple defect edge multi-scale focusing feature vectors are output, including: After the image to be detected is input into the convolution layer for convolution operation, convolution recursive feature extraction is performed through two cascaded convolution recursive fusion layers to obtain a first defect edge multi-scale focused feature vector; Using a convolutional recursive fusion layer to perform feature processing on the first defect edge multi-scale focused feature vector to generate a second defect edge multi-scale focused feature vector; After feature mapping is performed on the second defect edge multi-scale focused feature vector through the convolution recursive fusion layer, the feature vector is input into the spatial pyramid pooling layer for multi-scale pooling processing, and a third defect edge multi-scale focused feature vector is output.

3. The surface defect detection method based on feature focusing and refinement according to claim 2 is characterized in that: The convolution recursive fusion layer includes a convolution layer and a recursive convolution enhancement layer, the recursive convolution enhancement layer includes a recursive gated convolution sublayer, a feature segmentation sublayer and a recursive bottleneck block, and the recursive bottleneck block includes a plurality of cascaded recursive gated convolution sublayers; the processing process of the convolution recursive fusion layer includes: After the convolution input features input to the convolution recursive fusion layer are convolved by a convolution layer, recursive feature extraction is performed by a recursive gated convolution sublayer to generate convolution intermediate features; Using a feature segmentation sublayer to perform feature splitting on the convolution intermediate feature, and outputting a first convolution split feature and a second convolution split feature; After inputting the second convolution split feature into a plurality of cascaded recursive gated convolution sub-layers for continuous feature extraction, the feature is fused with the second convolution split feature to obtain a first convolution fusion feature; After extracting features step by step on the first convolutional fusion feature based on a plurality of cascaded recursive gated convolutional sub-layers, the first convolutional fusion feature is subjected to feature fusion to determine a second convolutional fusion feature; After channel splicing of the first convolution split feature, the first convolution fusion feature and the second convolution fusion feature, they are input into the recursive gated convolution sublayer for convolution processing to generate convolution output features.

4. The surface defect detection method based on feature focusing and refinement according to claim 1, characterized in that: The target reused fusion semantic network includes an attention layer, a depth-separable convolutional layer and a residual block; the multi-scale focused feature vectors of the defect edges are input into the target reused fusion semantic network for semantic enhancement, and a plurality of reused fusion semantic features are correspondingly generated, including: Input the multi-scale focused feature vectors of the defect edges into the cascaded attention layer and the depth-separable convolution layer respectively for channel space feature extraction and fusion, and obtain the first tomography-aware embedding feature, the second tomography-aware embedding feature and the third tomography-aware embedding feature accordingly; After upsampling the third tomographic-aware embedding feature, the feature fusion is performed with the second tomographic-aware embedding feature to generate a first semantic fusion feature; After upsampling the first semantic fusion feature, the feature fusion is performed with the first tomography-aware embedding feature to output a second semantic fusion feature; Performing feature extraction on the first semantic fusion feature and the second semantic fusion feature respectively through a residual block, and obtaining a residual feature and a first multiplexed fusion semantic feature respectively; After downsampling the first multiplexed fused semantic feature, perform feature fusion with the residual feature, perform feature processing through a residual block, and output a second multiplexed fused semantic feature; The residual feature is down-sampled and then fused with the third tomography-aware embedding feature, and feature extraction is performed via a residual block to output a third multiplexed fused semantic feature.

5. The surface defect detection method based on feature focusing and refinement according to claim 1, characterized in that: The target semantic-channel-space weighted refinement network includes a global average pooling layer, a global maximum pooling layer, a Conv_Silu layer, a Sigmoid activation function layer and a Softmax activation function layer; the processing process of the target semantic-channel-space weighted refinement network includes: The reused fusion semantic features input into the target semantic-channel-space weighted refinement network are respectively average pooled and maximum pooled through a global average pooling layer and a global maximum pooling layer, and average pooling features and maximum pooling features are generated accordingly; The average pooling feature and the maximum pooling feature are respectively convolved by the Conv_Silu layer, and then feature fusion is performed to output channel features; A Sigmoid activation function layer is used to extract the channel weight of the channel feature, and the channel weight is multiplied element by element by the multiplexed fusion semantic feature to obtain a channel weighted feature; Extracting the spatial weight of the multiplexed fusion semantic feature based on the Softmax activation function layer, and multiplying the spatial weight by the multiplexed fusion semantic feature element by element, and outputting the spatial weighted feature; The channel weighted features are fused with the space weighted features to generate semantic-channel-space weighted refined features.

6. The surface defect detection method based on feature focusing and refinement according to claim 1, characterized in that: The target feature focuses on the training process of the refinement defect detection model, including: Obtain defect-annotated low-resolution images; The defect-annotated low-resolution image is used to perform local model training on an initial edge-focused feature extraction network of an initial feature-focused and refined defect detection model, and when the first loss function value converges, an intermediate feature-focused and refined defect detection model is output; Based on the defect-annotated low-resolution image, the intermediate feature focusing and refinement defect detection model is trained as a whole until the second loss function value converges, and a target feature focusing and refinement defect detection model is determined.

7. The surface defect detection method based on feature focusing and refinement according to claim 6, characterized in that: The method of using the defect-annotated low-resolution image to perform local model training on the initial edge focus feature extraction network of the initial feature focus refinement defect detection model, and outputting an intermediate feature focus refinement defect detection model when the first loss function converges, comprises: In the initial edge focus feature extraction network of the initial feature focus refinement defect detection model, after convolution processing is performed on the defect annotated low-resolution image using a convolution layer, convolution recursive feature extraction is performed based on two cascaded convolution recursive fusion layers to generate training low-level features; Performing continuous feature processing on the training low-level features through two convolutional recursive fusion layers cascaded in the initial edge-focused feature extraction network to obtain training high-level features; Input the training low-level features and the training high-level features into a preset defect edge enhancement deep feature aggregation network for feature fusion enhancement, and output a training high-resolution image; Calculating a first loss function value based on the training high-resolution image and the defect-annotated high-resolution image corresponding to the defect-annotated low-resolution image; The model parameters in the initial edge focusing feature extraction network are iteratively optimized according to the first loss function value until the first loss function value converges to obtain an intermediate feature focusing refinement defect detection model.

8. The surface defect detection method based on feature focusing and refinement according to claim 7, characterized in that: The defect edge enhancement deep feature aggregation network includes an encoder and a decoder, the encoder includes an upsampling layer and a convolution layer with an activation function, and the decoder includes a convolution layer with an activation function and an image reconstruction layer; the inputting the training low-level features and the training high-level features into a preset defect edge enhancement deep feature aggregation network for feature fusion enhancement, and outputting a training high-resolution image, includes: Upsampling the training low-level features through an upsampling layer to obtain a first encoding feature; Using a convolutional layer with an activation function to perform feature mapping on the training high-level features to generate a second encoding feature; Perform channel concatenation of the first coding feature and the second coding feature, and output an image coding feature; Continuously extracting features of the image coding features through multiple convolutional layers with activation functions to obtain first decoding features; An image reconstruction layer is used to perform image super-resolution reconstruction based on the first decoded features to generate a training high-resolution image.

9. A surface defect detection system based on feature focusing and refinement, characterized in that: include: A feature input module is used to obtain an image to be detected and input the image to be detected into a preset target feature focusing and refinement defect detection model, wherein the target feature focusing and refinement defect detection model includes a target edge focusing feature extraction network, a target multiplexing fusion semantic network, a target semantic-channel-space weighted refinement network and a target detection layer; A feature extraction module, used to perform multi-scale feature extraction on the image to be detected through the target edge focusing feature extraction network, and output a plurality of defect edge multi-scale focusing feature vectors; A feature enhancement module, used for inputting the multi-scale focused feature vectors of each defect edge into the target multiplexing fusion semantic network for semantic enhancement, and correspondingly generating a plurality of multiplexing fusion semantic features; A feature refinement module, used to use the target semantic-channel-space weighted refinement network to perform feature filtering on each of the multiplexed fusion semantic features, and correspondingly determine a plurality of semantic-channel-space weighted refinement features; The defect detection module is used to perform defect location detection based on each of the semantic-channel-space weighted refinement features through the target detection layer, and output the defect detection result of the image to be detected.

10. A computer device, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the steps of the surface defect detection method based on feature focusing and refinement as described in any one of claims 1 to 8.

Images