Multi-scale target detection method based on detection picture data

Abstract

The invention relates to the technical field of defect detection, in particular to a multi-scale target detection method based on detection picture data, and aims to solve the problem of interference on edge target features during feature extraction. The method is applied to a target detection device, and the target detection device comprises an enhanced feature transfer backbone network, a cross-level feature fusion module and a multi-scale feature enhancement module. The method specifically comprises the following steps: acquiring an original defect detection picture, and respectively outputting a stage3 and a stage4; splitting the output of the stage4 to obtain a first feature map and a second feature map; carrying out feature fusion on the second feature map and the output of the stage3 level, and splitting to output a third feature map and a fourth feature map; and performing down-sampling on the fourth feature map, performing feature fusion on the fourth feature map and the first feature map, performing feature extraction, splicing the obtained output with the third feature map, performing uncertainty minimum query selection, inputting the selected result into a detection head, performing feature recognition and output, and completing image detection.

Description

Technical Field

[0001] This invention relates to the field of defect detection technology, and in particular to a multi-scale target detection method based on detection image data. Background Technology

[0002] Surface defect detection plays a crucial role in modern industrial production. Its purpose is to accurately identify various defects caused by numerous factors in the production of industrial products, such as breakage, stains, scratches, etc. Research on how to conduct efficient defect identification is of great significance. Compared with the huge costs of large-scale professional equipment such as X-ray and ultrasonic testing, computer vision-based detection technology is lower in cost and more suitable for widespread application in surface defect detection.

[0003] In the process of object detection, we may encounter the problem that the target to be detected is similar to the background, which leads to inaccurate detection. We also need to deal with the interference of object edges on the edge target features during feature extraction. Summary of the Invention

[0004] In view of this, the purpose of this invention is to propose a multi-scale target detection method based on detected image data, in order to solve the problems of the target being similar to the background and the interference of the object edge on the edge target features during feature extraction.

[0005] To achieve the above objectives, the present invention provides a multi-scale target detection method based on detected image data, using a target detection device, which includes a hybrid coding module and an enhanced feature transfer backbone network L-HGNetV2. The hybrid coding module integrates a cross-level feature fusion module CMF and a multi-scale feature enhancement module CDW.

[0006] A multi-scale target detection method based on detection image data specifically involves: acquiring the original defect detection image; performing stable feature transfer through an enhanced feature transfer backbone network L-HGNetV2; outputting from stages 3 and 4 of L-HGNetV2; passing the output of stage 4 through a transformer layer composed of a multi-head attention mechanism and multi-layer perception, then convolving to align the channels and splitting it into a first feature map and a second feature map; upsampling the second feature map and performing CMF feature fusion with the output of stage 3 in a cross-layer feature fusion module CMF, and then performing CDW feature extraction in a multi-scale feature enhancement module CDW, followed by channel alignment through convolution to split and output a third feature map and a fourth feature map; downsampling the fourth feature map and performing CMF feature fusion with the first feature map, then performing CDW feature extraction, and flattening and stitching the resulting output with the third feature map, performing minimum uncertainty query selection (MINQUES) and inputting it into the detection head for feature recognition and output, thus completing the image detection.

[0007] As a further improvement of this application, the enhanced feature transfer backbone network L-HGNetV2 includes an input head and a backbone network. The input head is used to receive the original defect detection image, and the backbone network processes the original defect detection image and outputs a feature map.

[0008] As a further improvement to this application, the input header is the collaborative module LOGSTEM.

[0009] As a further improvement of this application, the backbone network processes the original defect detection image and outputs a feature map. This includes first performing edge processing on the original defect detection image through a residual network layer, then performing convolutional Conv3×3 channel alignment, and then obtaining the output through two layers of Gaussian convolutional residual networks. The output is split into two parts: one part is processed through Conv3×3 convolution, and the other part is processed through MaxPool3×3 max pooling layers. These parts are then concatenated, and finally, the final output feature map is obtained by fusing Conv1×1.

[0010] As a further improvement of this application, the single-layer residual network includes a Gaussian-Laplace filter, wherein the Gaussian kernel of the Gaussian-Laplace filter is Conv7×7 and the Laplace kernel of the Gaussian-Laplace filter is Conv7×7.

[0011] As a further improvement to this application, the two-layer Gaussian convolution residual network includes a first layer of Gaussian convolution with a Conv9×9 layer and a second layer of Gaussian convolution with a Conv5×5 layer.

[0012] As a further improvement of this application, the cross-level feature fusion module (CMF) includes deep semantic features and shallow spatial features, and constructs a complementary mapping mechanism between the deep semantic features and shallow spatial features. The cross-level feature fusion module (CMF) first receives the feature map obtained by upsampling the second feature map and the feature map obtained by convolution from the stage3 output. The cross-level feature fusion module (CMF) then receives the feature map obtained by downsampling the first feature map and the fourth feature map. The received feature maps are concatenated and spliced ​​to fuse the feature information of the preceding and following levels to obtain Xc (H×W×C2). After feature fusion, context-aware calibration is performed through an adaptive squeezing and excitation mechanism to guide spatial focusing. ,Will Split into two parts and The original feature input is weighted using fusion weights, then connected, and finally channel-aligned using 1×1 convolution to output a feature map that fuses the feature information of the two layers.

[0013] As a further improvement of this application, the multi-scale feature enhancement module CDW receives feature maps output from the cross-layer feature fusion module CMF; the multi-scale feature enhancement module CDW includes a residual network; the multi-scale feature enhancement module CDW first receives feature maps output from the cross-layer feature fusion module CMF, then performs channel alignment on the received feature maps through convolution (Conv), and then splits them into two parts. One part is the main route that passes through the n-layer residual network, and the other part, through a residual connection mechanism, sums the input feature map and the output feature map element-wise with the output branches of each layer of the residual network. Finally, convolutional channel alignment is performed to obtain the output feature. The main route is composed of n layers of the residual network connected in series. Each layer of the residual network directly sums the output of the branches of other layers of the residual network element-wise through a branch. Finally, the output of the main route is connected with the summation result and then convolutional channel alignment is performed to obtain the output feature.

[0014] As a further improvement of this application, the residual network first obtains feature maps with scales C, C1, and 2C through Conv3×3 convolution and batch normalization, and then performs feature extraction through dilated convolutions with d=3, d=5, and d=1 respectively to achieve different receptive field ranges of 7×7, 11×11, and 3×3. Subsequently, it is spliced ​​and channel aligned, and then connected to the original residual network input to obtain the output of the residual network.

[0015] The beneficial effects of this invention are as follows: This invention proposes a multi-scale target detection method based on detected image data. It addresses the problem of similarity between the target and the background in surface defects. Through the cross-level feature fusion module CMF and the multi-scale feature enhancement module CDW, this network effectively establishes complex cross-level feature dependencies by adaptively integrating contextual feature sources, thereby improving the model's ability to model spatial structure and semantic relationships and strengthening feature extraction.

[0016] To address the interference of object edges on edge target features during feature extraction, the enhanced feature transfer backbone network L-HGNetV2 is used to promote the effective transfer of semantic information and spatial details, thereby improving the loss of features caused by edge interference in image downlink acquisition features. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only for this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a multi-scale target detection method based on detected image data according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of the cross-level feature fusion module (CMF) of a multi-scale target detection method based on detected image data according to an embodiment of the present invention.

[0020] Figure 3 This is a schematic diagram of the multi-scale feature enhancement module CDW of a multi-scale target detection method based on detected image data according to an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of the structure of L-HGNetV2, an enhanced feature transfer backbone network for a multi-scale target detection method based on detected image data, according to an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0023] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0024] like Figure 1-4 As shown, a multi-scale target detection method based on detection image data specifically involves: acquiring the original defect detection image; performing stable feature transfer through the enhanced feature transfer backbone network L-HGNetV2; outputting from stages 3 and 4 of the enhanced feature transfer backbone network L-HGNetV2; passing the output of stage 4 through a transformer layer composed of a multi-head attention mechanism and a multi-layer perception layer, then convolving to align the channels, and splitting it into a first feature map and a second feature map; upsampling the second feature map and performing CMF feature fusion with the output of stage 3 in the cross-layer feature fusion module CMF, and performing CDW feature extraction in the multi-scale feature enhancement module CDW, then performing channel alignment through convolution, and splitting it into a third feature map and a fourth feature map; downsampling the fourth feature map and performing CMF feature fusion with the first feature map, and performing CDW feature extraction, then flattening and stitching the resulting output with the third feature map, performing uncertainty minimum query selection, and inputting it into the detection head for feature recognition and output, thus completing the image detection.

[0025] The enhanced feature delivery backbone network L-HGNetV2 includes an input head and a backbone network. The input head receives the original defect detection image, and the backbone network processes the original defect detection image to output a feature map. The input head is a collaborative module LOGSTEM. The backbone network processes the original defect detection image to output a feature map, including first performing edge processing on the original defect detection image through a residual network layer, followed by convolutional Conv3×3 channel alignment, then passing through two layers of Gaussian convolutional residual networks to obtain the output. The output is split into two parts: one part passes through a Conv3×3 convolution, and the other part passes through a MaxPool3×3 max pooling layer, then they are concatenated, and finally fused through a Conv1×1 layer to obtain the final output feature map.

[0026] Specifically, the residual network includes a Gaussian-Laplace filter, wherein the Gaussian kernel is Conv7×7 and the Laplace kernel is Conv7×7.

[0027] A filter is defined as:

[0028]

[0029] Where e is a constant, x is the x-coordinate of the convolution kernel relative to the kernel center, y is the y-coordinate of the convolution kernel relative to the kernel center, and x and y represent the relative positions within the filter kernel window.

[0030] Gaussian filter It is given by the following formula:

[0031]

[0032] k×k represents the kernel size, and σ is the standard deviation, set to 1.0 for optimal edge detection.

[0033] Specifically, the two-layer Gaussian convolution residual network includes a first Gaussian convolution of Conv9×9 and a second Gaussian convolution of Conv5×5.

[0034] Traditional feature fusion methods only perform linear integration, ignoring the contextual relationships and dependencies between cross-layer features, thus limiting the network's ability to perceive multi-scale targets. The proposed cross-layer feature fusion module (CMF) constructs a complementary mapping mechanism between deep semantic features and shallow spatial features. This allows shallow information to effectively guide the construction of deep semantic representations, while deep semantics in turn reinforces the discriminative ability of shallow features. This achieves bidirectional optimization of semantic and localization information during feature transfer, thereby improving the detector's ability to capture salient features of targets.

[0035] The cross-level feature fusion module (CMF) includes deep semantic features and shallow spatial features, and constructs a complementary mapping mechanism between the deep semantic features and shallow spatial features. The CMF first receives a feature map obtained by upsampling the second feature map and a convolutional feature map output from stage 3. Then, the CMF receives a feature map obtained by downsampling the first and fourth feature maps. The received feature maps are concatenated to fuse the feature information from the preceding and following levels, resulting in Xc (H×W×C2). After feature fusion, context-aware calibration is performed through an adaptive squeezing and excitation mechanism to guide spatial focusing. ,Will Split into two parts and The original feature input is weighted using fusion weights, then connected, and finally channel-aligned using 1×1 convolution to output a feature map that fuses the feature information of the two layers.

[0036] During the fusion process, an adaptive convolution module is first introduced to achieve dynamic channel alignment between shallow and deep features. CMF has unconditional fusion capabilities across stages, which can significantly improve compatibility with different feature architectures, reduce reliance on a large number of manual preprocessing steps, and further improve the end-to-end learning efficiency and structural flexibility of the model.

[0037] If C1≠C2, then adjust Channel dimension:

[0038]

[0039] We first concatenate the feature inputs from the preceding and following layers to fuse the feature information from each layer:

[0040]

[0041] B represents the batch size, which is the number of samples input into the network at one time; C represents the number of channels; R refers to the channel dimension (Representation / dimensional expansion); X connects to form the concatenated feature map.

[0042] After fusing shallow and deep features, context-aware calibration is performed through an adaptive squeezing and excitation mechanism to guide spatial focusing. This mechanism emphasizes context-relevant key features. This targeted enhancement prioritizes information-rich patterns and strengthens the discriminative power of the final feature representation.

[0043] Global average pooling (Squeeze)

[0044] Perform global average pooling on each channel to obtain the channel description vector:

[0045]

[0046] Zc is the feature vector. : The pixel value of the feature map of the c-th channel at position (i,j);

[0047] The result is a vector:

[0048]

[0049] Z is the pooled feature vector, and Avg Pool represents average pooling.

[0050] The weights for each channel are generated using two fully connected layers:

[0051]

[0052] S represents the feature weights, σ ​​represents the sigmoid activation function, and W1 and W2 are the weight matrices of the fully connected layers; σ represents the sigmoid function.

[0053] Weighted Feature Map (Scale)

[0054] Transform the attention vector s into B×C×1×1 and weight it for the original input channels:

[0055]

[0056] Let c be the attention weight for the c-th channel;

[0057] Divided into two parts:

[0058]

[0059] : Input feature tensor, representing the result obtained by concatenating (enate) multiple features.

[0060] Split: Tensor splitting operation, splits a large tensor into multiple parts along the channel dimension.

[0061] C2: Channel size (this means that the spliced ​​channels are divided into two parts, each of which is C2 in size).

[0062] : The two sub-feature tensors after splitting.

[0063] Because the features derived from different levels have a large span, resulting in a low degree of correlation, this may lead to inaccurate matching of the target features. Therefore, we use fusion weights to weight the original features:

[0064]

[0065] Then combine and sum:

[0066]

[0067] A cross-branch bootstrapping mechanism is employed to achieve bidirectional feature enhancement. Specifically, each input branch in the module is weighted by calibration features provided by its complementary branch, thereby enabling dynamic interaction and information reconstruction between features. This complementary feature update mechanism not only strengthens the collaborative modeling capability of contextual cues but also effectively captures high-order feature dependencies that are ignored by traditional unidirectional or additive fusion strategies.

[0068] Output channel adjustment

[0069] If the number of channels after concatenation, C2, is not equal to ouc, perform another 1×1 convolution:

[0070]

[0071] ouc represents the number of downstream input channels, and y represents the output.

[0072] To enhance compatibility with downstream network layers, the module employs an adaptive convolutional module to adjust the dimensions of the fusion result, thereby improving computational efficiency while ensuring the integrity of feature representation. This structural design balances flexibility and practicality, enabling the module to have stronger generalization capabilities, adapt to diverse input formats, and generate more discriminative fusion features in complex visual tasks, significantly improving detection performance.

[0073] Existing feature extraction networks, by introducing multi-branch structures and multi-scale convolutional modules, have enhanced the diversity of feature representation and the receptive field to some extent. However, while multi-scale convolutional operations improve feature representation capabilities, they also significantly increase the computational burden and inference latency of the model during the feature extraction stage, potentially leading to feature redundancy and overfitting, thus limiting the improvement of detection robustness. To address this issue, this application proposes a multi-scale feature enhancement module (CDW), focusing on efficient encoding and representation capabilities after fusing features from previous layers. This structure is designed to balance multi-layer alignment, enhancing feature extraction capabilities while reducing model complexity.

[0074] The multi-scale feature enhancement module CDW receives feature maps output from the cross-layer feature fusion module CMF. The multi-scale feature enhancement module CDW includes a residual network. The multi-scale feature enhancement module CDW first receives feature maps output from the cross-layer feature fusion module CMF, then performs channel alignment on the received feature maps through convolution (Conv), and then splits them into two parts. One part's main path passes through n layers of the residual network, and the other part, through a residual connection mechanism, sums the input feature maps and output feature maps element-wise with the output branches of each layer of the residual network. Finally, convolutional channel alignment is performed to obtain the output features.

[0075] The residual network first obtains feature maps of scales C, C1, and 2C through Conv3×3 convolution, and then performs feature extraction through dilated convolutions of d=3, d=5, and d=1 respectively to achieve different receptive field ranges of 7×7, 11×11, and 3×3. Subsequently, it performs concatenation and channel alignment, and then connects it with the original residual network input to obtain the output of the residual network.

[0076] Specifically, in the first residual layer, 3×3 convolutions are used to generate feature maps of different sizes. In the second residual layer, we define multi-branch dilated convolutions and dilation rates:

[0077]

[0078]

[0079]

[0080] in, : Input base feature map, d represents the expansion rate, F1, F2, F3 represent the feature maps of the branches. : represents convolution. Similarly, we define d=3 and d=5.

[0081] We use dilated convolutions, with different dilation rates at the same level, to replace traditional pooling and convolutional layers. This expands the receptive field, making the model more advantageous in feature extraction.

[0082] splicing feature map:

[0083]

[0084] Append the output of each module to list Y:

[0085]

[0086] By introducing a multi-scale dilated convolutional structure, collaborative modeling of features within different receptive fields is achieved. Specifically, a convolution with a dilation rate of 1 corresponds to a standard 3×3 receptive field, focusing on extracting local details; a convolution with a dilation rate of 3 expands the receptive field to 7×7, used to capture mid-scale contextual relationships; and a convolution with a dilation rate of 5 further expands to an 11×11 receptive field, enhancing the ability to model wide-area semantic information. By applying these convolutional operations with different dilation rates in parallel, the module can model features from multiple scale perspectives and integrate local and global semantic information through feature map concatenation.

[0087] This multi-scale feature fusion strategy significantly enhances the model's ability to perceive and represent complex visual scenes. The module uses a residual connection mechanism to element-wise sum the input and output feature maps, effectively preserving low-level structural feature information. This design not only alleviates the gradient vanishing problem common in deep networks but also ensures that fine-grained information, such as edge contours and texture structures, is fully preserved during multi-scale feature extraction.

[0088] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of the invention (including the claims) is limited to these examples; within the framework of the invention, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of the different aspects of the invention as described above, which are not provided in the details for the sake of brevity.

[0089] This invention is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of the appended claims. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A multi-scale target detection method based on detected image data, characterized in that, The target detection device includes a hybrid coding module and an enhanced feature transfer backbone network L-HGNetV2. The hybrid coding module integrates a cross-level feature fusion module CMF and a multi-scale feature enhancement module CDW. The multi-scale target detection method based on detected image data is as follows: The original defect detection image is obtained and the features are stably transferred through the enhanced feature transfer backbone network L-HGNetV2. The stage3 and stage4 of the enhanced feature transfer backbone network L-HGNetV2 are output respectively. The output of stage4 passes through a transformer layer consisting of a multi-head attention mechanism and a multi-layer perception, and then passes through a convolutional alignment channel to split it into a first feature map and a second feature map. The second feature map is upsampled and the output of stage 3 is fused with CMF features in the cross-level feature fusion module CMF, and CDW features are extracted in the multi-scale feature enhancement module CDW. Then, channel alignment is performed through convolution, and the third and fourth feature maps are split and output. The fourth feature map is downsampled and fused with the first feature map using CMF feature fusion and CDW feature extraction. The output is then flattened and stitched together with the third feature map. The result is then selected using the minimum uncertainty query and input into the detection head for feature recognition and output, thus completing the detection of the image.

2. The multi-scale target detection method based on detected image data according to claim 1, characterized in that, The enhanced feature delivery backbone network L-HGNetV2 includes an input head and a backbone network. The input head is used to receive the original defect detection image, and the backbone network processes the original defect detection image and outputs a feature map.

3. The multi-scale target detection method based on detected image data according to claim 2, characterized in that, The input header is the LOGSTEM collaborative module.

4. The multi-scale target detection method based on detected image data according to claim 2, characterized in that, The backbone network processes the original defect detection image and outputs a feature map. This process includes first performing edge processing on the original defect detection image through a residual network layer, then performing convolutional Conv3×3 channel alignment, followed by two layers of Gaussian convolutional residual networks to obtain the output. The output is split into two parts: one part is processed through Conv3×3 convolution, and the other part is processed through a MaxPool3×3 max pooling layer. These parts are then concatenated, and finally, the final output feature map is obtained by fusing Conv1×1.

5. The multi-scale target detection method based on detected image data according to claim 4, characterized in that, The single-layer residual network includes a Gaussian-Laplace filter, wherein the Gaussian kernel of the Gaussian-Laplace filter is Conv7×7, and the Laplace kernel of the Gaussian-Laplace filter is Conv7×7.

6. The multi-scale target detection method based on detected image data according to claim 4, characterized in that, The two-layer Gaussian convolution residual network includes a first Gaussian convolution of Conv9×9 and a second Gaussian convolution of Conv5×5.

7. The multi-scale target detection method based on detected image data according to claim 1, characterized in that, The cross-level feature fusion module CMF includes deep semantic features and shallow spatial features, and constructs a complementary mapping mechanism between deep semantic features and shallow spatial features. The cross-level feature fusion module CMF first receives the feature map obtained by upsampling the second feature map and the feature map output by stage3 after convolution; The cross-level feature fusion module (CMF) then receives a feature map obtained by downsampling the first feature map and the fourth feature map; The received feature maps are concatenated and stitched together to fuse feature information from different layers to obtain Xc. After feature fusion, context-aware calibration is performed through an adaptive squeezing and excitation mechanism to guide spatial focusing. ,Will Split into two parts and The original feature input is weighted using fusion weights, then connected, and finally channel-aligned using 1×1 convolution to output a feature map that fuses the feature information of the two layers.

8. The multi-scale target detection method based on detected image data according to claim 1, characterized in that, The multi-scale feature enhancement module CDW receives feature maps output from the cross-layer feature fusion module CMF; The multi-scale feature enhancement module CDW includes a residual network; The multi-scale feature enhancement module CDW first receives the feature map output from the cross-level feature fusion module CMF, then performs channel alignment on the received feature map through convolution Conv, and then splits it into two parts: one part is the main route, and the other part is connected to the output branch of each layer of the residual network through the residual connection mechanism to sum the input feature map and the output feature map element by element. Finally, convolution channel alignment is performed to obtain the output feature. The main route is composed of n layers of residual networks connected in series. Each layer of the residual network outputs directly through a branch. Finally, the output of the main route is connected with the summation result and then convolutional channels are aligned to obtain the output features.

9. The multi-scale target detection method based on detected image data according to claim 8, characterized in that, The residual network first obtains feature maps of scales C, C1, and 2C through Conv3×3 convolution and batch normalization. Then, it performs feature extraction through dilated convolutions of d=3, d=5, and d=1 respectively to achieve different receptive field ranges of 7×7, 11×11, and 3×3. Subsequently, it performs concatenation and channel alignment, and then connects it with the original residual network input to obtain the output of the residual network.

Images