A target detection method and system based on multi-scale feature fusion and dual-domain enhancement

Abstract

The application discloses a target detection method and system based on multi-scale feature fusion and dual-domain enhancement, and relates to the technical field of computer vision. In view of the problems of large defect scale span, similar texture and low contrast in an industrial scene, a space depth conversion convolution (SPDConv) is introduced in the feature extraction stage, fine-grained information in the downsampling process is reserved by rearrangement; a double convolution parallel processing module (DC2f) is designed in the feature fusion stage, local modeling and cross-channel interaction are considered by parallel group convolution and point-by-point convolution; a dual-domain attention enhancement module (SEDAM) is introduced, micro defect features are highlighted and background interference is suppressed by multi-scale pre-enhancement and space-channel parallel weighting. The application effectively balances the detection accuracy and reasoning speed, and is suitable for real-time online detection of metal surface defects.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a method and system for detecting small targets and low-contrast defects in industrial settings. Background Technology

[0002] In modern industrial systems, especially in high-end manufacturing fields such as precision electronics and aerospace, the quality control of metal surfaces faces severe challenges. Defects such as cracks, pitting, and scratches not only vary in size but are also often hidden in complex background textures. This low-contrast characteristic places extremely high demands on the robustness of detection algorithms.

[0003] Existing deep learning detection methods have the following shortcomings when handling this type of task:

[0004] 1. Traditional stride convolution downsampling is prone to irreversible loss of fine-grained information, leading to missed detection of minute defects;

[0005] 2. In pursuing lightweight design, existing feature fusion modules often weaken cross-channel interaction capabilities and struggle to distinguish similar textures;

[0006] 3. Conventional attention mechanisms are usually used in the deep layers of the network and often only focus on a single domain, making it difficult to effectively suppress noise and highlight weak targets in complex backgrounds.

[0007] Therefore, there is an urgent need for a robust method and system for detecting surface defects in workpieces that is not prone to missing minute defects, can fully extract defect features, and is robust. Summary of the Invention

[0008] This invention aims to solve the problems of easy omission of small defects and insufficient feature extraction in the prior art, and provides a target detection method based on multi-scale feature fusion and dual-domain enhancement.

[0009] The technical solution adopted by this invention to achieve the above objectives is: a target detection method based on multi-scale feature fusion and dual-domain enhancement, comprising the following steps:

[0010] Acquire an image of the industrial surface to be detected, and input the industrial surface image into a pre-constructed target detection network;

[0011] Feature extraction is performed on the industrial surface image through the backbone network; during the downsampling stage of the backbone network, a spatial depth transformation convolution module is used for downsampling to rearrange the spatial dimension information of the feature map to the channel dimension.

[0012] The extracted features are fused at multiple scales through the neck network; in the feature fusion stage, a dual convolution parallel processing module is used, which includes parallel group convolution branches and pointwise convolution branches, which are used to extract local structural features and cross-channel semantic features respectively, and the two outputs are fused.

[0013] During feature extraction or fusion, the features are weighted by a dual-domain attention enhancement module, which includes parallel multi-scale convolutional pre-enhancement paths, spatial attention paths, and channel attention paths. The final weighted features are generated through additive fusion.

[0014] The features processed by the neck network are input into the detection head, which outputs the target's category and location information.

[0015] The spatial depth transformation convolution module performs the following steps:

[0016] Obtaining the input feature map ,in For feature map size, Number of channels;

[0017] The input feature map is spatially rearranged to depthwise, and then downsampled according to a preset factor. Slice the sub-blocks in the spatial dimension and stitch them together to the channel dimension to obtain the intermediate feature map.

[0018] Perform sequential processing on the intermediate feature maps Convolution compresses channels, and Non-staggered convolution is used to obtain the downsampled output feature map.

[0019] The dual-convolution parallel processing module performs the following steps:

[0020] The input features are divided into two parallel computation paths;

[0021] The first path uses a kernel size of Group convolutions are used to extract local spatial structure features;

[0022] The second path uses a kernel size of Pointwise convolution is used to perform linear combinations across channels;

[0023] The output feature maps of the first path and the second path are added element by element to obtain the fused features.

[0024] The dual-domain attention enhancement module performs the following steps:

[0025] First, the input features are pre-enhanced using multi-scale convolutional blocks. Each multi-scale convolutional block contains at least three parallel branches with convolutional kernels of different sizes. The outputs of each branch are summed and then channel integration is performed to obtain the enhanced features.

[0026] The enhanced features are input into the spatial attention path and the channel attention path, respectively;

[0027] The spatial attention path calculates the average and maximum values ​​of the feature maps in the channel dimension, concatenates them, and then generates a spatial weight map through a large kernel convolution; the large kernel convolution is a k×k convolution, where k≥5;

[0028] The channel attention path performs global average pooling and global max pooling on the feature map, and then generates channel weight vectors after being processed by a multi-layer perceptron or convolutional layer, respectively.

[0029] The final output feature is the sum of the results obtained by performing element-wise multiplication operations on the enhanced feature with the spatial weight map and the channel weight vector, respectively.

[0030] A target detection system based on multi-scale feature fusion and dual-domain enhancement includes:

[0031] Image acquisition module, used to acquire images of the industrial surface to be inspected;

[0032] An object detection network for detecting object category and location based on industrial surface images; including:

[0033] The backbone network is used to extract features from the industrial surface image. Convolutions are replaced with spatial depth transformation convolutions, and the C2F module is replaced with a dual-domain attention enhancement module. The spatial depth transformation convolution is used for downsampling to rearrange the spatial dimension information of the feature map to the channel dimension. The dual-domain attention enhancement module is used to perform weighted processing on the features. The dual-domain attention enhancement module includes parallel multi-scale convolutional pre-enhancement paths, spatial attention paths, and channel attention paths, which generate the final weighted features through additive fusion.

[0034] The neck network is used to perform multi-scale fusion of extracted features; wherein, the C2F module is replaced by a dual convolution parallel processing module; the dual convolution parallel processing module includes parallel group convolution branches and pointwise convolution branches, which are used to extract local structural features and cross-channel semantic features, respectively, and the two outputs are fused.

[0035] The detection output module is used to obtain the category and location box of the defect based on the characteristics output by the neck network.

[0036] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned target detection method based on multi-scale feature fusion and dual-domain enhancement.

[0037] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the target detection method based on multi-scale feature fusion and dual-domain enhancement.

[0038] The present invention has the following beneficial effects and advantages:

[0039] 1. Improved small target detection capability: SPDConv effectively preserves the fine-grained features of shallow layers.

[0040] 2. Enhanced feature robustness: The combination of DC2f and SEDAM enables the model to accurately identify defects even in environments with low contrast and strong texture interference.

[0041] 3. Maintains high efficiency: While improving accuracy, this invention controls the number of parameters through structural optimization, making it suitable for deployment on edge computing devices. Experiments show that on the NEU-DET dataset, this method achieves a 3.5% improvement over the baseline model mAP50, and the inference speed meets real-time requirements. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the overall process and network architecture of the method provided in the embodiments of the present invention;

[0043] Figure 2 This is a schematic diagram of the spatial depth transformation convolution (SPDConv) in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the structure of the dual convolution parallel processing module (DC2f) in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the dual-domain attention enhancement module (SEDAM) in an embodiment of the present invention;

[0046] Figure 5 This is a comparison chart showing the effectiveness of the embodiments of the present invention in defect detection tasks. Detailed Implementation

[0047] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0048] Spatial Depth Transform Convolution (SPDConv) is introduced during the downsampling stage of the backbone network. Through Space-to-Depth rearrangement, the spatial information of the high-resolution feature map is transferred to the channel dimension, followed by non-stretched convolution. This operation avoids the information loss associated with traditional stretched convolutions and preserves edge and texture features of minor defects. A dual-convolution parallel processing module (DC2f) is designed for the feature fusion stage. This module executes in parallel. Group convolution and Pointwise convolutions are used, with the former capturing local spatial structure and the latter handling cross-channel semantic interaction; the outputs of both are added together. This design enhances feature discriminativity while reducing computational cost. A dual-domain attention enhancement module (SEDAM) is also designed. This module first uses multi-scale convolutions for context pre-enhancement, then performs parallel attention weighting of the spatial and channel domains, and finally outputs the result using additive fusion. This module can simultaneously highlight both the location and category semantic information of defects.

[0049] A target detection method based on multi-scale feature fusion and dual-domain enhancement includes the following steps:

[0050] (1) Acquire the image to be detected and input the image to be detected into the pre-constructed target detection network; the image to be detected is an RGB image of a metal surface.

[0051] (2) The image to be detected is extracted through the backbone network. During the downsampling stage of the backbone network, the spatial depth transformation convolution module (SPDConv) is used to downsample and rearrange the spatial dimension information of the feature map to the channel dimension to retain fine-grained features.

[0052] (3) Multi-scale fusion of the extracted features is performed through the neck network. In the feature fusion stage, a dual convolution parallel processing module (DC2f) is used. The dual convolution parallel processing module includes parallel group convolution branches and point-by-point convolution branches, which are used to extract local structural features and cross-channel semantic features, respectively, and the two outputs are fused.

[0053] (4) In the feature extraction or fusion process, the features are weighted by the dual-domain attention enhancement module (SEDAM). The dual-domain attention enhancement module includes parallel multi-scale convolutional pre-enhancement path, spatial attention path and channel attention path, and generates the final weighted features through additive fusion.

[0054] (5) Input the processed features into the detection head and output the target's category and location information. Defect categories include: cracks, inclusions, patches, pitting, scratches, and rolled-in oxide scale.

[0055] The training process of the object detection network includes:

[0056] S1: Obtain image data based on the steel plate surface defect dataset, and label the defect targets by category and location;

[0057] S2: Construct a training sample set, input the samples into the target detection network for forward propagation, and obtain the target category prediction value and the location prediction value;

[0058] S3: Calculate the loss function based on the error between the predicted value and the true label. The loss function includes classification loss and location regression loss.

[0059] S4: Perform backpropagation based on the loss function to update the network parameters until the model training is complete.

[0060] The specific processing steps of the Spatial Depth Transform Convolutional Module (SPDConv) include: acquiring the input feature map. ,in For feature map size, The number of channels; the input feature map is rearranged from space to depth according to a preset downsampling factor. Slice the sub-blocks in the spatial dimension and stitch them together to the channel dimension to obtain the intermediate feature map. Perform sequential processing on the intermediate feature maps Convolution compresses channels, and The non-stride convolution (stride=1) yields the downsampled output feature map.

[0061] The processing procedure of the dual convolutional parallel processing module (DC2f) includes: dividing the input features into two parallel computation paths; the first path uses a kernel size of... Group convolution is used to extract local spatial structure features; the second path uses a kernel size of [missing information]. Point-wise convolution is used to perform linear combination across channels; the output feature maps of the first and second paths are added element-wise to obtain fused features.

[0062] The specific structure and processing of the dual-domain attention enhancement module (SEDAM) are as follows: First, the input features are pre-enhanced using multi-scale convolutional blocks, each containing parallel branches with at least three convolutional kernels of different sizes. The outputs of each branch are summed and then channel-integrated. The pre-enhanced features are then input into the spatial attention path and the channel attention path, respectively. The spatial attention path calculates the average and maximum values ​​of the feature map in the channel dimension, concatenates them, and generates a spatial weight map through a large kernel convolution (k×k convolution, k≥5). The channel attention path performs global average pooling and global max pooling on the feature map, and then processes it through a multi-layer perceptron or convolutional layer to generate a channel weight vector. The final output feature is the product of the original input feature and the spatial weight map, plus the product of the original input feature and the channel weight vector. The cross-attention mechanism achieves spatial semantic correspondence through feature projection between the query feature and the supporting features, and its output maintains the original two-dimensional spatial structure, exhibiting position sensitivity.

[0063] Example 1:

[0064] like Figure 1 As shown, the SDS-YOLO model proposed in this invention is based on an improved YOLOv8 architecture and mainly includes a backbone network, a neck network, and a head. The object detection network includes the backbone network, neck network, and head unit in sequence. In the backbone network, convolutions are replaced with spatial depth transformation convolutions, and C2F modules are replaced with dual-domain attention enhancement modules. In the neck network, C2F modules are replaced with dual-convolution parallel processing modules.

[0065] Structure Preservation Downsampling (SPDConv)

[0066] like Figure 2 As shown, for small defects on metal surfaces, traditional convolutional stride sampling leads to information loss. This embodiment uses SPDConv in the first two downsampling stages of the backbone network. Assume the input features are...

[0067] The downsampling factor is First, a Space-to-Depth transformation is performed to transform the feature map from...

[0068] Rearranged as At this point, spatial information is losslessly transferred to the channel dimension. Subsequently, through...

[0069] Convolution compresses the channels, then passes through Feature extraction is performed using convolutions with a stride of 1. The formula is as follows:

[0070]

[0071]

[0072] This step eliminates sampling bias, preserving both low-frequency and high-frequency information in the image.

[0073] Parallel and efficient convolutional fusion (DC2f)

[0074] like Figure 3 As shown, in the feature fusion module of the Neck section, this embodiment uses a DC2f structure instead of the original C2f. DC2f contains two parallel paths within a bottleneck block:

[0075] Path A: Group convolution focuses on extracting local spatial features and reduces the number of parameters.

[0076] Path B: Point-wise convolution focuses on the linear fusion of information between channels.

[0077] The output is This dual-path parallel design takes into account both the receptive field and channel interaction, effectively solving the problem of false detection caused by texture similarity.

[0078] Multi-scale dual-domain attention (SEDAM)

[0079] like Figure 4 As shown, this embodiment embeds a SEDAM module before feature transmission at different scales.

[0080] Preprocessing: The input features first go through a... The convolution is then split into three parallel convolution branches (kernel sizes of 5, 7, and 9), and the three outputs are summed to capture contextual information at different scales.

[0081] Dual-domain weighting:

[0082] Spatial Path (MSSA): Performs channel-dimensional mean and max pooling on the features, then concatenates them. Convolution generates a spatial mask.

[0083] Channel Path (MSCA): Global Mean and Max pooling are performed on the features, and channel weights are generated through MLP processing.

[0084] Fusion: .in, Indicates the features after preprocessing. Indicates a spatial mask. This indicates the channel weight.

[0085] Additive fusion allows the model to enhance both spatial and channel features simultaneously, making it particularly suitable for enhancing low-contrast defects.

[0086] Example 2:

[0087] To verify the effectiveness of this invention, tests were conducted on the NEU-DET steel surface defect dataset. The experimental environment used an NVIDIA RTX 3080 GPU. Figure 5 As shown.

[0088] The method of this invention achieves an mAP50 of 78.5% and an mAP95 of 44.7%, representing improvements of 3.5 and 2.7 percentage points respectively compared to the original YOLOv8n. Simultaneously, the number of parameters is reduced by 1.8%, and the inference speed reaches 108.7 FPS, demonstrating that this method possesses industrial real-time detection capabilities while maintaining high accuracy.

[0089] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A target detection method based on multi-scale feature fusion and dual-domain enhancement, characterized in that, Includes the following steps: Acquire an image of the industrial surface to be detected, and input the industrial surface image into a pre-constructed target detection network; Feature extraction is performed on the industrial surface image through the backbone network; during the downsampling stage of the backbone network, a spatial depth transformation convolution module is used for downsampling to rearrange the spatial dimension information of the feature map to the channel dimension. The extracted features are fused at multiple scales using a neck network. In the feature fusion stage, a dual convolution parallel processing module is used, which includes parallel group convolution branches and pointwise convolution branches, which are used to extract local structural features and cross-channel semantic features, respectively, and the two outputs are fused. During feature extraction or fusion, the features are weighted by a dual-domain attention enhancement module, which includes parallel multi-scale convolutional pre-enhancement paths, spatial attention paths, and channel attention paths. The final weighted features are generated through additive fusion. The features processed by the neck network are input into the detection head, which outputs the target's category and location information.

2. The target detection method based on multi-scale feature fusion and dual-domain enhancement according to claim 1, characterized in that, The spatial depth transformation convolution module performs the following steps: Obtaining the input feature map ,in For feature map size, Number of channels; The input feature map is spatially rearranged to depthwise, and then downsampled according to a preset factor. Slice the sub-blocks in the spatial dimension and stitch them together to the channel dimension to obtain the intermediate feature map. ; Perform sequential processing on the intermediate feature maps Convolution compresses channels, and Non-staggered convolution is used to obtain the downsampled output feature map.

3. The target detection method based on multi-scale feature fusion and dual-domain enhancement according to claim 1, characterized in that, The dual-convolution parallel processing module performs the following steps: The input features are divided into two parallel computation paths; The first path uses a kernel size of Group convolutions are used to extract local spatial structure features; The second path uses a kernel size of Pointwise convolution is used to perform linear combinations across channels; The output feature maps of the first path and the second path are added element by element to obtain the fused features.

4. The target detection method based on multi-scale feature fusion and dual-domain enhancement according to claim 1, characterized in that, The dual-domain attention enhancement module performs the following steps: First, the input features are pre-enhanced using multi-scale convolutional blocks. Each multi-scale convolutional block contains at least three parallel branches with convolutional kernels of different sizes. The outputs of each branch are summed and then channel integration is performed to obtain the enhanced features. The enhanced features are input into the spatial attention path and the channel attention path, respectively; The spatial attention path calculates the average and maximum values ​​of the feature maps in the channel dimension, concatenates them, and then generates a spatial weight map through a large kernel convolution; the large kernel convolution is a k×k convolution, where k≥5; The channel attention path performs global average pooling and global max pooling on the feature map, and then generates channel weight vectors after being processed by a multi-layer perceptron or convolutional layer, respectively. The final output feature is the sum of the results obtained by performing element-wise multiplication operations on the enhanced feature with the spatial weight map and the channel weight vector, respectively.

5. A target detection system based on multi-scale feature fusion and dual-domain enhancement, characterized in that, include: Image acquisition module, used to acquire images of the industrial surface to be inspected; An object detection network for detecting object category and location based on industrial surface images; including: The backbone network is used to extract features from the industrial surface image. Convolutions are replaced with spatial depth transformation convolutions, and the C2F module is replaced with a dual-domain attention enhancement module. The spatial depth transformation convolution is used for downsampling to rearrange the spatial dimension information of the feature map to the channel dimension. The dual-domain attention enhancement module is used to perform weighted processing on the features. The dual-domain attention enhancement module includes parallel multi-scale convolutional pre-enhancement paths, spatial attention paths, and channel attention paths, which generate the final weighted features through additive fusion. The neck network is used to perform multi-scale fusion of extracted features; wherein, the C2F module is replaced by a dual convolution parallel processing module; the dual convolution parallel processing module includes parallel group convolution branches and pointwise convolution branches, which are used to extract local structural features and cross-channel semantic features, respectively, and the two outputs are fused. The detection output module is used to obtain the category and location box of the defect based on the characteristics output by the neck network.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements a target detection method based on multi-scale feature fusion and dual-domain enhancement as described in any one of claims 1 to 4.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements a target detection method based on multi-scale feature fusion and dual-domain enhancement as described in any one of claims 1 to 4.

Images