A target detection system and method based on hierarchical feature fusion

Abstract

This invention relates to a target detection system and method based on hierarchical feature fusion. It employs an efficient hybrid encoder to extract edge features from the feature image obtained by the first feature extraction of the backbone network, generating two edge feature maps at different scales. Then, it performs attention-based internal scale feature interaction on the feature image obtained by the m-th feature extraction of the backbone network to obtain an interactive feature map. Subsequently, it performs hierarchical feature fusion on the two edge feature maps, the interactive feature map, and all feature images obtained by the second to (m-1)-th feature extractions of the backbone network to obtain a fused feature map. This enables edge feature extraction from the feature image obtained by the first feature extraction of the backbone network, significantly enhancing the system's ability to capture both global and local information while overcoming noise interference.

Description

Technical Field

[0001] This invention relates to the field of computer modeling and systems technology, and more specifically, to a target detection system and method based on hierarchical feature fusion. Background Technology

[0002] In target detection tasks in complex environments, especially in industrial settings, accurate detection of small targets and their edge features is crucial. While traditional target detection methods can handle most scenarios, they often fall short in capturing details and edge information, leading to reduced detection accuracy in complex backgrounds.

[0003] To address this, existing technologies disclose a lightweight multi-scale feature calibration method for small target detection. This method uses a feature calibration mechanism to sample the most representative information from deep features for each pixel location and perform spatial alignment, thereby alleviating the problem of losing key details due to spatial misalignment or information dilution during feature fusion. Furthermore, it enhances the detection performance of this method for small targets by using a high-resolution feature layer model.

[0004] However, in some complex industrial scenarios, due to challenges such as background noise interference and blurred edge details, the lightweight model detection system based on this small target detection method has a weak ability to cope with background noise interference in complex environments, making it difficult to meet the detection performance requirements in specific scenarios, resulting in poor deployment effect of the final detection system. Summary of the Invention

[0005] The technical problem to be solved by this invention is how to overcome the technical deficiency of lightweight model detection systems based on existing small target detection methods in dealing with background noise interference in complex environments. To overcome this technical deficiency, this invention provides a target detection system and method based on hierarchical feature fusion, specifically including a target detection system based on hierarchical feature fusion and a target detection method based on hierarchical feature fusion.

[0006] This invention provides a target detection system based on hierarchical feature fusion, comprising:

[0007] The backbone network is configured to perform m feature extractions on the image under test through compound convolution to obtain m feature images. After each of the first (m-1) feature extractions, the obtained feature image is output and used as the object of the next feature extraction operation. After the m-th feature extraction, the obtained feature image is output.

[0008] A high-efficiency hybrid encoder, communicating with the backbone network, is configured to perform edge feature extraction on the feature image obtained from the first feature extraction of the backbone network to generate two edge feature maps at different scales, and to perform attention-based internal scale feature interaction on the feature image obtained from the m-th feature extraction of the backbone network to obtain an interactive feature map; subsequently, hierarchical feature fusion is performed on the two edge feature maps, the interactive feature map, and all feature images obtained from the second to (m-1)-th feature extractions of the backbone network to obtain a fused feature map;

[0009] The decoding module, which communicates with the efficient hybrid encoder, is configured to sequentially perform uncertainty minimum query selection and multiple decoder layer iterations on the fused feature map to generate corresponding category prediction results and bounding box prediction results for each object of the minimum query selection.

[0010] Where m > 3.

[0011] The hierarchical feature fusion-based target detection system disclosed in this invention employs a backbone network, an efficient hybrid encoder, and a decoding module. The efficient hybrid encoder extracts edge features from the feature image obtained from the first feature extraction by the backbone network to generate two edge feature maps at different scales. Then, attention-based internal scale feature interaction is performed on the feature image obtained from the m-th feature extraction by the backbone network to obtain an interactive feature map. Subsequently, the two edge feature maps, the interactive feature map, and all feature images obtained from the second to (m-1)-th feature extractions by the backbone network are hierarchically fused to obtain a fused feature map. This enables edge feature extraction within the feature image obtained from the first feature extraction by the backbone network. Furthermore, the attention-based internal scale feature interaction effectively limits the scope of self-attention operations to a single-scale feature map, effectively reducing memory usage while better utilizing the rich semantic information in high-level features. This improves the overall system's ability to detect and recognize targets and enhances its ability to cope with background noise interference in complex environments. In addition, the hierarchical attention mechanism of hierarchical feature fusion efficiently integrates multiple features, significantly enhancing the system's ability to capture global and local information, thereby improving the system's understanding of complex patterns. Subsequently, through the selection of queries with minimal uncertainty and iterative optimization of multiple decoder layers, the representation of object queries can be gradually optimized, generating corresponding category prediction and bounding box prediction results for each object query.

[0012] In one possible implementation, the high-efficiency hybrid encoder includes:

[0013] The edge feature extraction module communicates with the backbone network and is configured to extract edge features from the feature image obtained by the first feature extraction of the backbone network to generate two edge feature maps at different scales.

[0014] The feature interaction module communicates with the backbone network and is configured to perform attention-based internal scale feature interaction on the feature image obtained by the m-th feature extraction of the backbone network to obtain an interactive feature map.

[0015] The first-level feature fusion network, which communicates with both the feature interaction module and the backbone network, is configured to perform hierarchical feature fusion on all feature images obtained from the second to (m-1)th feature extractions of the interaction feature map and the backbone network to obtain a primary fusion feature map.

[0016] The second-level feature fusion network communicates with both the edge feature extraction module and the first-level feature fusion network. It is configured to perform (m-2) hierarchical feature fusion on the two edge feature maps and the primary fusion feature map to obtain (m-2) secondary fusion feature maps.

[0017] The edge fusion device, which communicates with the edge feature extraction module, the first-level feature fusion network and the second-level feature fusion network, is configured to perform multi-scale edge feature fusion with each of the two edge feature maps and each of the secondary fusion feature maps and the primary fusion feature map to obtain (m-1) final fusion feature maps.

[0018] The stitching unit, which communicates with both the edge fusion device and the decoding module, is configured to perform channel stitching on all the final fusion feature maps to obtain the fusion feature map.

[0019] The efficient hybrid encoder, equipped with the aforementioned structure and functionality, performs intra-scale interactions on the deepest features through a feature interaction module. By limiting the scope of self-attention operations to a single-scale feature map, it effectively reduces memory consumption while better utilizing the rich semantic information in high-level features, thereby improving the encoder's ability to detect and recognize targets. Through the setup of a first-level and a second-level feature fusion network, it achieves efficient fusion of multiple features using a hierarchical attention mechanism, significantly enhancing the model's ability to capture both global and local information. Furthermore, with the cooperation of the edge feature extraction module and the edge fusion device, edge features are fused at multiple scales, allowing the efficient hybrid encoder to focus more on significant edge regions while effectively suppressing background interference.

[0020] In one possible implementation, the edge feature extraction module is configured to perform the following steps:

[0021] A1: Perform Sobel convolution processing on the feature image obtained from the initial feature extraction of the backbone network to obtain the first feature map;

[0022] A2: Perform max pooling on the first feature map to obtain the second feature map;

[0023] A3: Perform max pooling on the second feature map to obtain the third feature map, and simultaneously perform convolution on the second feature map to obtain the fourth feature map;

[0024] A4: Perform convolution processing on the combination of the third feature map and the fourth feature map to obtain the fifth feature map;

[0025] A5: Perform max pooling on the third feature map to obtain one of the edge feature maps, and then perform convolution on the combination of this edge feature map and the fifth feature map to obtain another edge feature map.

[0026] The edge feature extraction module, which operates in the manner described above, first extracts edge information from the feature image obtained by the initial feature extraction of the backbone network through Sobel convolution. The Sobel convolution method can also calculate the edge magnitude of each channel. Then, the spatial resolution is gradually reduced through multiple max pooling operations to generate edge feature maps of different scales, so that the entire system can simultaneously focus on the edge feature information of the target at both the global and local scales.

[0027] In one possible implementation, the first hierarchical feature fusion network is a network structure composed of (m-2) first fusion devices connected in series. The first fusion device is a network structure composed of a first convolutional unit and a hierarchical feature fusion module connected in series. The first convolutional unit in the first fusion device communicates with the feature interaction module. All the hierarchical feature fusion modules in the first hierarchical feature fusion network communicate with the backbone network.

[0028] The first convolutional unit is configured to sequentially perform a convolution operation with a kernel size of 1×1, batch normalization, and SiLU activation function processing on its input image to obtain a convolutional image; wherein, the input image of the first convolutional unit in the first fusion device is the interactive feature map.

[0029] The hierarchical feature fusion module is configured to execute a hierarchical feature fusion algorithm with two inputs and one output to obtain an output image; wherein, the output image of the hierarchical feature fusion module in the first fusion device at the end is the primary fusion feature map.

[0030] In one possible implementation, the second hierarchical feature fusion network is a network structure composed of (m-2) second fusion devices connected in series. The second fusion device is a network structure composed of a second convolutional unit and the hierarchical feature fusion module connected in series. The second convolutional unit in the first second fusion device communicates with the hierarchical feature fusion module in the last first fusion device.

[0031] The hierarchical feature fusion module in the first second fusion device communicates with the first convolutional unit in the last first fusion device, the hierarchical feature fusion module in the last second fusion device communicates with the edge feature extraction module, and the hierarchical feature fusion modules in the remaining second fusion devices communicate with the first convolutional unit in a one-to-one manner.

[0032] The second convolutional unit is configured to sequentially perform a 3×3 convolution operation, batch normalization, and SiLU activation function processing on its input image to obtain its output image; wherein, the input image of the second convolutional unit in the first second fusion device is the primary fusion feature map;

[0033] In the second-level feature fusion network, the output image of all the hierarchical feature fusion modules is a secondary fusion feature map.

[0034] The combination of the first-level feature fusion network and the second-level feature fusion network, which have the above structure and functions, not only forms a hierarchical feature fusion detection network, but also, through the hierarchical attention mechanism completed by setting up multiple hierarchical feature fusion modules, can more efficiently fuse multiple features, significantly enhancing the network's ability to capture global and local information, thereby improving the understanding of complex patterns by the efficient hybrid encoder.

[0035] In one possible implementation, the hierarchical feature fusion module is configured to perform a hierarchical feature fusion algorithm comprising the following steps:

[0036] B1: The following two processes are performed in parallel:

[0037] The input 1 is convolved to obtain the convolution result of input 1;

[0038] Perform convolution on input 2 to obtain the convolution result of input 2;

[0039] B2: The following three processes will be performed in parallel:

[0040] The input 1 convolution result is mapped using a local attention mechanism to obtain a first attention image, and the input 1 convolution result is also mapped using a global attention mechanism to obtain a second attention image. Then, the first attention image and the second attention image are concatenated by channels to obtain a first concatenated image.

[0041] The convolution result of input 1 is summed with the convolution result of input 2, and then the summation result is convolved to obtain a combined convolution image;

[0042] The input 2 convolution result is mapped using a local attention mechanism to obtain a third attention image, and the input 2 convolution result is also mapped using a global attention mechanism to obtain a fourth attention image. Then, the third attention image and the fourth attention image are concatenated by channels to obtain a second concatenated image.

[0043] B3: The first stitched image, the convolutional image, and the second stitched image are stitched together by channels to obtain a final stitched image. Then, the final stitched image is sequentially subjected to convolution processing, reparameterized convolution processing, and convolution processing to obtain the output image of the hierarchical feature fusion module.

[0044] For the hierarchical feature fusion module in the first fusion device, its input 1 is the output image of the first convolutional unit in the first fusion device, which is the same as the hierarchical feature fusion module, and its input 2 is a feature image obtained by the second to (m-1)th feature extraction of the backbone network.

[0045] In the second-level feature fusion network, input 1 of the hierarchical feature fusion module is the output image of the second convolutional unit in the second fusion device, which is the same as the hierarchical feature fusion module, and input 2 is the output image of a certain first convolutional unit or two edge feature maps output by the edge feature extraction module.

[0046] The hierarchical feature fusion module, operating as described above, efficiently fuses multiple input features through a hierarchical attention mechanism, significantly enhancing its ability to capture both global and local information. Step B2, by utilizing local and global attention mechanisms to process image features at different scales, effectively overcomes the limitations of single-scale features and precisely adjusts the importance of features at each layer, thereby better fusing information at different scales. Simultaneously, multi-level feature fusion significantly improves the ability to capture both global and local information, effectively handling complex environments and diverse targets while reducing computational overhead.

[0047] In one possible implementation, the edge fusion device is a network structure consisting of (m-1) multi-scale edge feature fusion modules connected in parallel. All of the multi-scale edge feature fusion modules communicate with the edge feature extraction module, and all of the multi-scale edge feature fusion modules also communicate one-to-one with the hierarchical feature fusion module in the first fusion device at the end and with all the hierarchical feature fusion modules in the second hierarchical feature fusion network. The stitching unit communicates with all of the multi-scale edge feature fusion modules simultaneously.

[0048] The multi-scale edge feature fusion module is configured to execute a dual-input single-output multi-scale edge feature fusion algorithm to transform the two edge feature maps with the primary fusion feature map or one of the secondary fusion feature maps into the final fusion feature map.

[0049] In one possible implementation, the multi-scale edge feature fusion module is configured to perform a multi-scale edge feature fusion algorithm comprising the following steps:

[0050] C1: Perform channel-by-channel stitching of the two edge feature maps with the primary fusion feature map or one of the secondary fusion feature maps to obtain a channel-stitched image;

[0051] C2: The channel stitched image is sequentially subjected to convolution processing with a kernel size of 1×1, convolution processing with a kernel size of 3×3, and convolution processing with a kernel size of 1×1 to obtain the final fusion feature map.

[0052] The multi-scale edge feature fusion module first concatenates feature maps from different scales along the channel dimension to generate a channel-stitched image that incorporates edge information. Next, a 1×1 convolutional layer compresses the number of channels, reducing computation and eliminating redundant information. Then, a 3×3 convolutional layer extracts local features from the fused features, further enhancing the expressive power of edge information. Finally, a 1×1 convolution adjusts the number of channels to integrate the extracted features and generate the final output feature map, fully utilizing multi-scale edge information and improving information reliability.

[0053] Another technical solution of the present invention is to provide a target detection method based on hierarchical feature fusion, comprising the following steps:

[0054] Step 1: Optimize the model parameters of the network formed by the backbone network, the efficient hybrid encoder and the decoding module in series using the IPIoU loss function;

[0055] Step 2: The backbone network performs m feature extractions on the image under test using compound convolution to obtain m feature images. After each of the first (m-1) feature extractions, the obtained feature image is output and used as the object of the next feature extraction operation. After the m-th feature extraction, the obtained feature image is output.

[0056] Step 3: Edge features are extracted from the feature image obtained by the first feature extraction of the backbone network using an efficient hybrid encoder to generate two edge feature maps at different scales. Then, attention-based internal scale feature interaction is performed on the feature image obtained by the m-th feature extraction of the backbone network to obtain an interactive feature map. Subsequently, hierarchical feature fusion is performed on the two edge feature maps, the interactive feature map, and all feature images obtained by the second to (m-1)-th feature extractions of the backbone network to obtain a fused feature map.

[0057] Step 4: The decoding module sequentially performs minimum uncertainty query selection and multiple decoder layer iterations on the fused feature map to generate corresponding category prediction results and bounding box prediction results for each object selected by minimum uncertainty query selection.

[0058] The hierarchical feature fusion-based target detection method disclosed in this invention first optimizes the model parameters of the network formed by the sequential concatenation of the backbone network, the efficient hybrid encoder, and the decoding module using the IPIoU loss function to improve regression accuracy, significantly accelerate the network's convergence speed, and enhance the model's generalization ability. Then, the efficient hybrid encoder extracts edge features from the feature image obtained from the first feature extraction of the backbone network to generate two edge feature maps at different scales. Next, attention-based internal scale feature interaction is performed on the feature image obtained from the m-th feature extraction of the backbone network to obtain an interactive feature map. Subsequently, the two edge feature maps, the interactive feature map, and all feature images obtained from the second to (m-1)-th feature extractions of the backbone network are hierarchically fused to obtain a fused feature map. This achieves edge feature extraction from the feature image obtained from the first feature extraction of the backbone network. Furthermore, the attention-based internal scale feature interaction effectively limits the scope of self-attention operations to a single-scale feature map, significantly reducing the method's memory footprint while better utilizing the rich semantic information in high-level features. This enhances the method's ability to detect and recognize targets and improves its ability to handle background noise interference in complex environments. In addition, the hierarchical attention mechanism of hierarchical feature fusion efficiently integrates multiple features, significantly enhancing the method's ability to capture global and local information, thereby improving its understanding of complex patterns. Finally, through uncertainty-minimum query selection and iterative optimization across multiple decoder layers, the representation of object queries can be progressively optimized, generating corresponding category predictions and bounding box predictions for each object query. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of a target detection system structure based on hierarchical feature fusion disclosed in an embodiment of the present invention;

[0060] Figure 2 This is a flowchart illustrating the operation of the edge feature extraction module disclosed in this embodiment of the invention.

[0061] Figure 3 This is a flowchart illustrating the operation of the hierarchical feature fusion module disclosed in this embodiment of the invention.

[0062] Figure 4 This is a flowchart illustrating the operation of the multi-scale edge feature fusion module disclosed in this embodiment of the invention.

[0063] Figure 5 This is a schematic diagram of the loss function disclosed in the embodiments of the present invention.

[0064] Explanation of reference numerals in the attached figures:

[0065] 1. Backbone Network; 2. High-Efficiency Hybrid Encoder; 21. Edge Feature Extraction Module; 22. Feature Interaction Module; 23. Multi-Scale Edge Feature Fusion Module; 24. Separation Unit; 25. First Convolutional Unit; 26. Hierarchical Feature Fusion Module; 27. Second Convolutional Unit; 3. Decoding Module. Detailed Implementation

[0066] First, those skilled in the art should understand that these embodiments are merely used to explain the technical principles of the embodiments of this application and are not intended to limit the scope of protection of the embodiments of this application. Those skilled in the art can make adjustments as needed to adapt to specific application scenarios.

[0067] In the description of the embodiments of this application, it should be noted that, unless otherwise explicitly specified and limited, the term "forming a communication link structure" means that the multiple communication elements or modules involved form a network structure or network link structure through communication connection. Communication or communication connection means that there is information transmission between the first feature and the second feature. This information transmission can be unidirectional or bidirectional. The communication connection can be realized by electrical connection of wires, radio connection, electrical connection of electromagnetic media (such as optical fiber, semiconductor), communication realized by channel, etc.

[0068] In the embodiments of this application, unless otherwise expressly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0069] The present application will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0070] See Figures 1-5 This application discloses a target detection system based on hierarchical feature fusion. Figure 1 This is a schematic diagram of the target detection system. The target detection system includes a backbone network 1, an efficient hybrid encoder 2, and a decoding module 3. The efficient hybrid encoder 2 communicates with the backbone network 1, and the decoding module 3 communicates with the efficient hybrid encoder 2.

[0071] In this object detection system, the backbone network 1 is configured to perform m feature extractions on the target image using compound convolution to obtain m feature images. After each of the first (m-1) feature extractions, the obtained feature image is output and used as the object for the next feature extraction. After the m-th feature extraction, the obtained feature image is output. Let's denote the m output terminals (input terminals denoted by S1) of the backbone network 1 as S2, S3, ..., Sm+1, that is, the feature image obtained by the backbone network 1 after t feature extractions is output through output terminal Sm+1, 1≤t≤m, m>3. In this embodiment, the backbone network 1 uses a lightweight ResNet18 network architecture, which has four output terminals, denoted as S2, S3, S4, and S5, respectively. Figure 1 As shown.

[0072] See Figure 1In this target detection system, the efficient hybrid encoder 2 is configured to perform edge feature extraction on the feature image obtained by the first feature extraction of the backbone network 1 to generate two edge feature maps of different scales, and perform attention-based internal scale feature interaction on the feature image obtained by the m-th feature extraction of the backbone network 1 to obtain an interaction feature map; then, hierarchical feature fusion is performed on the two edge feature maps, the interaction feature map and all feature images obtained by the second to (m-1)-th feature extraction of the backbone network 1 to obtain a fused feature map. Specifically, in this embodiment, the high-efficiency hybrid encoder 2 includes an edge feature extraction module 21, a feature interaction module 22, a first-level feature fusion network, a second-level feature fusion network, an edge fusion device, and a stitching unit 24. The edge feature extraction module 21 communicates with the backbone network 1, the feature interaction module 22 communicates with the backbone network 1, the first-level feature fusion network communicates with both the feature interaction module 22 and the backbone network 1, the second-level feature fusion network communicates with both the edge feature extraction module 21 and the first-level feature fusion network, the edge fusion device communicates with the edge feature extraction module 21, the first-level feature fusion network, and the second-level feature fusion network, and the stitching unit 24 communicates with both the edge fusion device and the decoding module 3.

[0073] In the high-efficiency hybrid encoder 2, the edge feature extraction module 21 is configured to extract edge features from the feature image obtained from the initial feature extraction of the backbone network 1 to generate two edge feature maps at different scales. See also Figure 2 In this embodiment, the edge feature extraction module 21 is configured to perform the following steps: A1: Perform Sobel convolution processing on the feature image obtained from the initial feature extraction of the backbone network 1 to obtain a first feature map, such as... Figure 2 As shown. A2: Perform max pooling on the first feature map to obtain the second feature map. A3: Perform max pooling on the second feature map to obtain the third feature map, and simultaneously perform convolution on the second feature map to obtain the fourth feature map. A4: Perform convolution on the combination of the third and fourth feature maps to obtain the fifth feature map. A5: Perform max pooling on the third feature map to obtain one of the edge feature maps, and simultaneously perform convolution on the combination of this edge feature map and the fifth feature map to obtain the other edge feature map.

[0074] In the high-efficiency hybrid encoder 2, the feature interaction module 22 is configured to perform attention-based internal scale feature interaction on the feature image obtained from the m-th feature extraction of the backbone network 1 to obtain an interactive feature map. In this embodiment, the feature interaction module 22 performs attention-based internal scale feature interaction on the feature image output by the output terminal S5 of the ResNet18 network architecture to obtain an interactive feature map. Figure 2 The blue and yellow boxes in the image represent different forms of feature maps.

[0075] In the efficient hybrid encoder 2, the first-level feature fusion network is configured to hierarchically fuse the interactive feature map and all feature images obtained from the second to (m-1)th feature extractions of the backbone network 1 to obtain a primary fused feature map. See [link to documentation]. Figure 1 In this embodiment, the first-level feature fusion network is a network structure composed of (m-2) first fusion devices connected in series. Each first fusion device is a network structure composed of a first convolutional unit 25 and a hierarchical feature fusion module 26 connected in series. The first convolutional unit 25 in the first fusion device communicates with the feature interaction module 22, and all hierarchical feature fusion modules 26 in the first-level feature fusion network communicate with the backbone network 1. The first convolutional unit 25 is configured to sequentially perform a convolution operation with a kernel size of 1×1, batch normalization processing, and SiLU activation function processing on its input image to obtain a convolutional image. The input image of the first convolutional unit 25 in the first fusion device is the interaction feature map. For the remaining first convolutional units 25, their input image is the output image of the hierarchical feature fusion module 26 in the previous first fusion device of the first convolutional unit 25.

[0076] In the efficient hybrid encoder 2, the second-level feature fusion network is configured to perform (m-2) hierarchical feature fusion of two edge feature maps with the primary fused feature map to obtain (m-2) secondary fused feature maps. See also Figure 1 In this embodiment, the second-level feature fusion network is a network structure composed of (m-2) second fusion devices connected in series. Each second fusion device is a network structure composed of a second convolutional unit 27 and a hierarchical feature fusion module 26 connected in series. The second convolutional unit 27 in the first fusion device communicates with the hierarchical feature fusion module 26 in the last fusion device. The hierarchical feature fusion module 26 in the first fusion device communicates with the first convolutional unit 25 in the last fusion device, and the hierarchical feature fusion module 26 in the last fusion device communicates with the edge feature extraction module 21. The remaining hierarchical feature fusion modules 26 in the other second fusion devices communicate with the first convolutional unit 25 in a one-to-one manner. The second convolutional unit 27 is configured to sequentially perform a 3×3 convolution operation, batch normalization, and SiLU activation function processing on its input image to obtain its output image; wherein, the input image of the second convolutional unit 27 in the first second fusion device is the primary fusion feature map, and for the second convolutional units 27 in other second fusion devices, its input image is the output image of the hierarchical feature fusion module 26 in the previous second fusion device in which the second convolutional unit 27 is located.

[0077] The hierarchical feature fusion module 26 is configured to execute a hierarchical feature fusion algorithm with two inputs and one output to obtain an output image. In the first hierarchical feature fusion network, the output image of the hierarchical feature fusion module 26 in the last fusion device is a primary fusion feature map. For the hierarchical feature fusion module 26 in the first fusion device, its input 1 is the output image of the first convolutional unit 25 in the same fusion device as the hierarchical feature fusion module 26, and its input 2 is a feature image obtained from the second to (m-1)th feature extraction of the backbone network 1. In the second hierarchical feature fusion network, the output images of all hierarchical feature fusion modules 26 are secondary fusion feature maps, and its input 1 is the output image of the second convolutional unit 27 in the same fusion device as the hierarchical feature fusion module 26, and its input 2 is the output image of the first convolutional unit 25 or the output image of the edge feature extraction module 21 (i.e., two edge feature maps).

[0078] To better capture feature information at each level, this embodiment also arranges the input 2 of the hierarchical feature fusion module 26 in the first-level feature fusion network. The first fusion device at the beginning is designated as the first fusion device, and the first fusion devices are sorted in ascending order from 1 to the end. Then, the input 2 of the hierarchical feature fusion module 26 in the first fusion device is designated as the feature image output from the Sm output terminal of the backbone network 1; the input 2 of the hierarchical feature fusion module 26 in the second fusion device is designated as the feature image output from the Sm-1 output terminal of the backbone network 1; ..., the input 2 of the hierarchical feature fusion module 26 in the t-th fusion device is designated as the feature image output from the Sm-t+1 output terminal of the backbone network 1; ..., the input 2 of the hierarchical feature fusion module 26 in the (m-2)-th fusion device is designated as the feature image output from the S3 output terminal of the backbone network 1.

[0079] See Figure 3In this embodiment, the hierarchical feature fusion module 26 is configured to execute a hierarchical feature fusion algorithm comprising the following steps: B1: Parallel processing of the following two processes: First, performing convolution processing on input 1 to obtain the convolution result of input 1; Second, performing convolution processing on input 2 to obtain the convolution result of input 2. B2: Parallel processing of the following three processes: First, performing local attention mechanism mapping processing on the convolution result of input 1 to obtain a first attention image, and also performing global attention mechanism mapping processing on the convolution result of input 1 to obtain a second attention image, and then concatenating the first attention image and the second attention image by channels to obtain a first concatenated image; Second, summing the convolution result of input 1 and the convolution result of input 2, and then performing convolution processing on the obtained summing result to obtain a combined convolution image; Third, performing local attention mechanism mapping processing on the convolution result of input 2 to obtain a third attention image, and also performing global attention mechanism mapping processing on the convolution result of input 2 to obtain a fourth attention image, and then concatenating the third attention image and the fourth attention image by channels to obtain a second concatenated image. B3: The first stitched image, the convolutional image, and the second stitched image are stitched together by channels to obtain the final stitched image. Then, the final stitched image is sequentially subjected to convolution processing, reparameterized convolution processing, and convolution processing to obtain the output image of the hierarchical feature fusion module 26.

[0080] To conserve computing resources, this embodiment also adopts a shared strategy for the hierarchical feature fusion module 26, specifically as follows: Figure 1 As shown, the second fusion device at the top is also called the first second fusion device. The second fusion devices are sorted in ascending order from 1 to the bottom. Then, the first second fusion device and the (m-3)th first fusion device share a hierarchical feature fusion module 26, the second second fusion device and the (m-4)th first fusion device share a hierarchical feature fusion module 26, ..., the tth second fusion device and the (mt-1)th first fusion device share a hierarchical feature fusion module 26, ..., the (m-3)th second fusion device and the first first fusion device share a hierarchical feature fusion module 26.

[0081] See Figure 1In the high-efficiency hybrid encoder 2, the edge fusion device is configured to perform multi-scale edge feature fusion with each of the two edge feature maps and each of the secondary fusion feature maps and the primary fusion feature map to obtain (m-1) final fusion feature maps. The stitching unit 24 is configured to perform channel stitching on all the final fusion feature maps to obtain a fusion feature map. The edge fusion device is a network structure consisting of (m-1) multi-scale edge feature fusion modules 23 connected in parallel. All multi-scale edge feature fusion modules 23 communicate with the edge feature extraction module 21, and all multi-scale edge feature fusion modules 23 also communicate one-to-one with the hierarchical feature fusion module 26 in the first fusion device at the end and with all hierarchical feature fusion modules 26 in the second hierarchical feature fusion network. The stitching unit 24 communicates with all multi-scale edge feature fusion modules 23 simultaneously.

[0082] See Figure 4 The multi-scale edge feature fusion module 23 is configured to perform a dual-input single-output multi-scale edge feature fusion algorithm to fuse two edge feature maps ( Figure 4 The edge feature fusion module 23 is configured to perform a multi-scale edge feature fusion algorithm comprising the following steps: C1: concatenating two edge feature maps with the primary fusion feature map or one of the secondary fusion feature maps to obtain a channel-stitched image; C2: sequentially performing convolution processing with a kernel size of 1×1, a kernel size of 3×3, and a kernel size of 1×1 on the channel-stitched image to obtain the final fusion feature map.

[0083] See Figure 1 In this target detection system, the decoding module 3 is configured to perform uncertainty minimum query selection and multiple decoder layer iterations on the fused feature map in sequence to generate corresponding category prediction results and bounding box prediction results for each object selected by the minimum query selection.

[0084] The target detection method corresponding to the target detection system based on hierarchical feature fusion in this embodiment will be further disclosed below. The method includes the following steps:

[0085] See Figure 5 Step 1: Optimize the model parameters of the network formed by the backbone network 1, the high-efficiency hybrid encoder 2 and the decoding module 3 in series using the IPIoU loss function.

[0086] Traditional IoU-based loss functions may cause anchor boxes to expand unreasonably to cover the target during regression, resulting in a complex and slow convergence path. Furthermore, existing focusing mechanisms have a monotonic attention to sample quality and cannot effectively distinguish between anchor boxes of different quality. The Powerful-IoU (PIoU) loss function significantly accelerates convergence and effectively alleviates the anchor box expansion problem by guiding anchor boxes along a more direct and efficient regression path. The Powerful-IoU loss function effectively solves the anchor box expansion problem by introducing a target size-adaptive penalty factor, as shown in the following formula:

[0087] ,

[0088] in, It is the distance between the corresponding boundary points of the predicted bounding box and the target bounding box, such as Figure 5 As shown, and These are the width and height of the target bounding box, respectively.

[0089] Corresponding adjustment function as follows:

[0090] ,

[0091] when When the value is large (poor anchor frame quality), ,lead to At this point, the gradient is relatively small, which can effectively suppress the interference of low-quality samples on the training process; when When it is in the medium range, make The derivative of the gradient reaches its peak, significantly increasing the gradient and thus accelerating the regression efficiency of medium-quality anchor frames; when When the size is small (anchor box is close to the target box), ,at this time The gradient decreases accordingly, avoiding oscillations during the optimization process and ensuring convergence stability. The final loss function can be expressed as:

[0092] ,

[0093] However, the Powerful-IoU loss function exhibits instability in scenarios with dense small targets and sparse large targets. Therefore, this embodiment incorporates the idea of ​​the Inner-IoU loss function into the Powerful-IoU loss function, designing the IPIoU loss function. This loss function overcomes the limitations of traditional IoU loss functions in convergence speed and generalization ability, effectively improving the model's detection performance in these extreme scenarios. A schematic diagram of the IPIoU loss function is shown below. Figure 5 As shown in the figureb and b gt Represents a designated point. For point b coordinates and Points b The length and width of the rectangle it contains; For point b gt coordinates and Points b gt The length and width of a certain rectangle.

[0094] The IPIoU loss function introduces an auxiliary bounding box into the calculation of the Inner-IoU loss function, using a scaling factor ratio=0.7 to control the size of the auxiliary box, thereby achieving adaptive gradient control for the regression samples. Its calculation process (the meanings of the relevant parameters are as follows) is explained in the diagram. Figure 5 As shown below:

[0095] , ,

[0096] , ,

[0097] , ,

[0098] , ,

[0099] ,

[0100] ,

[0101] ,

[0102] In summary, the final IPIoU loss function is as follows:

[0103] .

[0104] Step 2: The backbone network 1 performs m feature extractions on the image under test using compound convolution to obtain m feature images. After each of the first (m-1) feature extractions, the obtained feature image is output and used as the object of the next feature extraction operation. After the mth feature extraction, the obtained feature image is output.

[0105] Step 3: The feature image obtained from the first feature extraction of the backbone network 1 is processed by the efficient hybrid encoder 2 to extract edge features to generate two edge feature maps of different scales. The feature image obtained from the m-th feature extraction of the backbone network 1 is then processed by attention-based internal scale feature interaction to obtain an interactive feature map. Subsequently, the two edge feature maps, the interactive feature map and all feature images obtained from the second to (m-1)-th feature extractions of the backbone network 1 are hierarchically fused to obtain a fused feature map.

[0106] Step 4: The fused feature map is sequentially subjected to minimum uncertainty query selection and multiple decoder layer iterations through decoding module 3 to generate corresponding category prediction results and bounding box prediction results for each object selected by minimum uncertainty query selection.

[0107] The technical performance of the hierarchical feature fusion-based object detection system in this embodiment will be described in detail below. The experimental platform in this embodiment uses Ubuntu 22.04.4 LTS as the operating system and an Intel® Xeon® Platinum 8336C CPU@2.3GHz×128 processor and an NVIDIA GeForce RTX 4090 graphics card. The software environment uses CUDA 11.3 and PyTorch 2.2.2 as the deep learning framework. RT-DETR-R18 is used as the baseline model to verify the effectiveness of the object detection system. Furthermore, the object detection system is trained from scratch with 300 iterations, a batch size of 8, and the AdamW optimization function. All other hyperparameters use the default settings of the baseline model.

[0108] To verify the performance of the object detection system, this embodiment compares it with currently popular object detection models. Experimental results on the FactorySafeDet dataset are shown in Table 1:

[0109] Table 1. Performance comparison with other models on the FactorySafeDet dataset.

[0110]

[0111] Popular object detection models used include YOLOv8m, YOLOv11m, YOLOv9-c, YOLOv10m, RetinaNet, Faster-RCNN, and Mask R-CNN. Experimental results show that the mAP of this object detection system is [missing information]. 50 The mAP and recall rates reached 98.3% and 97.2% respectively, compared to the baseline model. 50The mAP and recall rates improved by 2.1 and 3.7 percentage points respectively, significantly reducing the false negative rate. Compared to the suboptimal model, mAP... 50 The proposed target detection system outperforms existing models in all metrics, demonstrating superior performance. The recall rate is 1.2 percentage points and 2.4 percentage points higher, respectively.

[0112] Furthermore, to verify the performance of the proposed IPIoU loss function, this embodiment compares the IPIoU loss function with other popular loss functions. All comparisons are based on the baseline model, with only the loss function being changed. Performance comparisons with other loss functions on the FactorySafeDet dataset are shown in Table 2.

[0113] Table 2 shows the performance comparison with other loss functions on the FactorySafeDet dataset.

[0114]

[0115] This embodiment compares mainstream loss functions including Powerful-IoU, Inner-IoU, MPDIoU, Focal Loss, and Focaler-IoU. Experimental results show that the proposed IPIoU loss function improves both recall and mAP50. The IPIoU loss function incorporates the Inner-IoU concept into Powerful-IoU, effectively guiding anchor boxes to regress along a more direct and efficient path. By introducing auxiliary bounding boxes, it achieves adaptive gradient control for regressed samples, overcoming the limitations of traditional IoU loss in convergence speed and generalization ability, thereby improving the model's detection performance in these extreme scenarios.

[0116] To verify the effectiveness of each module of the target detection system, ablation experiments were conducted on the FactorySafeDet test set for all modules of the target detection system, either individually or in different combinations. The results are shown in Table 3.

[0117] Table 3 Ablation experimental results on the FactorySafeDet dataset

[0118]

[0119] The experimental results show that by introducing a multi-scale edge perception module consisting of an edge feature extraction module 21 and a multi-scale edge feature fusion module 23, the model's mAP is improved. 50The recall and accuracy improved by 1.2 percentage points and 2.7 percentage points, respectively. This indicates that the module combination effectively enhances the edge feature extraction capability and achieves efficient fusion of multi-scale edge features, enabling it to focus more on significant edge regions while effectively suppressing background interference. Based on this, by adding the hierarchical feature fusion module 26, the model's mAP... 50 Compared to the baseline model, the recall and accuracy improved by 1.0 percentage point and 2.3 percentage point, respectively. This indicates that through multi-level feature fusion, the hierarchical feature fusion module 26 can better capture detailed and global information, thereby improving the model's ability to understand complex patterns. Furthermore, by introducing the IPIoU loss function, the model further improved mAP. 50 The recall and accuracy rates improved by 2.1 and 3.7 percentage points respectively compared to the baseline model. This indicates that the introduction of the auxiliary bounding box mechanism effectively improved the model's detection performance in some extreme scenarios. Therefore, the object detection system proposed in this embodiment significantly improves the detection performance in complex environments with only a slight increase in the number of parameters and computational cost.

[0120] In summary, the hierarchical feature fusion-based target detection system disclosed in this embodiment, by setting up a backbone network, an efficient hybrid encoder, and a decoding module, uses the efficient hybrid encoder to extract edge features from the feature image obtained from the first feature extraction of the backbone network to generate two edge feature maps at different scales. Then, it performs attention-based internal scale feature interaction on the feature image obtained from the m-th feature extraction of the backbone network to obtain an interactive feature map. Subsequently, it performs hierarchical feature fusion on the two obtained edge feature maps, the interactive feature map, and all feature images obtained from the second to (m-1)-th feature extractions of the backbone network to obtain a fused feature map. This achieves edge feature extraction from the feature image obtained from the first feature extraction of the backbone network. Furthermore, the attention-based internal scale feature interaction effectively limits the scope of self-attention operations to a single-scale feature map, effectively reducing memory usage while better utilizing the rich semantic information in high-level features, thereby improving the model's ability to detect and recognize targets and enhancing the overall system's ability to cope with background noise interference in complex environments. Furthermore, the hierarchical attention mechanism of hierarchical feature fusion efficiently integrates multiple features, significantly enhancing the system's ability to capture global and local information, thereby improving the system's understanding of complex patterns. Subsequently, through uncertainty-minimum query selection and iterative optimization of multiple decoder layers, the representation of object queries can be gradually optimized, generating corresponding category predictions and bounding box predictions for each object query.

[0121] In the description of the embodiments of this application, it should be noted that the terms "inner" and "outer" and other terms indicating direction or positional relationship are based on the direction or positional relationship shown in the drawings. This is only for the convenience of description and does not indicate or imply that the device or component must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of this application.

[0122] In the description of this application, the references to terms such as "an embodiment," "some embodiments," "in this embodiment," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0123] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A target detection system based on hierarchical feature fusion, characterized in that, include: The backbone network (1) is configured to perform m feature extractions on the image under test through compound convolution to obtain m feature images. After each of the first (m-1) feature extractions, the obtained feature image is output and used as the object of the next feature extraction operation. After the mth feature extraction, the obtained feature image is output. The high-efficiency hybrid encoder (2), which communicates with the backbone network (1), is configured to perform edge feature extraction on the feature image obtained by the first feature extraction of the backbone network (1) to generate two edge feature maps of different scales, and to perform attention-based internal scale feature interaction on the feature image obtained by the m-th feature extraction of the backbone network (1) to obtain an interactive feature map; subsequently, the two edge feature maps, the interactive feature map, and all feature images obtained by the second to (m-1)-th feature extraction of the backbone network (1) are hierarchically fused to obtain a fused feature map; The decoding module (3), which communicates with the efficient hybrid encoder (2), is configured to perform uncertainty minimum query selection and multiple decoder layer iterations on the fused feature map in sequence to generate corresponding category prediction results and bounding box prediction results for each object of the minimum query selection; Where m > 3.

2. The target detection system based on hierarchical feature fusion according to claim 1, characterized in that, The high-efficiency hybrid encoder (2) includes: The edge feature extraction module (21) communicates with the backbone network (1) and is configured to perform edge feature extraction on the feature image obtained from the first feature extraction of the backbone network (1) to generate two edge feature maps of different scales. The feature interaction module (22) communicates with the backbone network (1) and is configured to perform attention-based internal scale feature interaction on the feature image obtained by the m-th feature extraction of the backbone network (1) to obtain an interactive feature map; The first-level feature fusion network, which communicates with the feature interaction module (22) and the backbone network (1), is configured to perform hierarchical feature fusion on all feature images obtained from the second to (m-1)th feature extraction of the interaction feature map and the backbone network (1) to obtain a primary fusion feature map; The second-level feature fusion network communicates with both the edge feature extraction module (21) and the first-level feature fusion network. It is configured to perform (m-2) hierarchical feature fusion on the two edge feature maps and the primary fusion feature map to obtain (m-2) secondary fusion feature maps. The edge fusion device, which communicates with the edge feature extraction module (21), the first-level feature fusion network and the second-level feature fusion network, is configured to perform multi-scale edge feature fusion with each of the two edge feature maps and each of the secondary fusion feature maps and the primary fusion feature map to obtain (m-1) final fusion feature maps. The splicing unit (24), which communicates with both the edge fusion device and the decoding module (3), is configured to perform channel splicing on all the final fusion feature maps to obtain the fusion feature map. 3.The target detection system based on hierarchical feature fusion according to claim 2, characterized in that, The edge feature extraction module (21) is configured to perform the following steps: A1: Perform Sobel convolution processing on the feature image obtained from the first feature extraction of the backbone network (1) to obtain the first feature map; A2: Perform max pooling on the first feature map to obtain the second feature map; A3: Perform max pooling on the second feature map to obtain the third feature map, and simultaneously perform convolution on the second feature map to obtain the fourth feature map; A4: Perform convolution processing on the combination of the third feature map and the fourth feature map to obtain the fifth feature map; A5: Perform max pooling on the third feature map to obtain one of the edge feature maps, and then perform convolution on the combination of this edge feature map and the fifth feature map to obtain another edge feature map.

4. The target detection system based on hierarchical feature fusion according to claim 2 or 3, characterized in that, The first-level feature fusion network is a network structure composed of (m-2) first fusion devices connected in series. The first fusion device is a network structure composed of a first convolutional unit (25) and a hierarchical feature fusion module (26) connected in series. The first convolutional unit (25) in the first fusion device communicates with the feature interaction module (22). All the hierarchical feature fusion modules (26) in the first-level feature fusion network communicate with the backbone network (1). The first convolutional unit (25) is configured to sequentially perform a convolution operation with a kernel size of 1×1, batch normalization, and SiLU activation function processing on its input image to obtain a convolutional image; wherein, the input image of the first convolutional unit (25) in the first fusion device is the interactive feature map. The hierarchical feature fusion module (26) is configured to execute a hierarchical feature fusion algorithm with two inputs and one output to obtain an output image; wherein, the output image of the hierarchical feature fusion module (26) in the first fusion device at the end is the primary fusion feature map.

5. The hierarchical feature fusion based object detection system of claim 4, wherein, The second-level feature fusion network is a network structure composed of (m-2) second fusion devices connected in series. The second fusion device is a network structure composed of a second convolution unit (27) and the hierarchical feature fusion module (26) connected in series. The second convolution unit (27) in the first second fusion device communicates with the hierarchical feature fusion module (26) in the last first fusion device. The hierarchical feature fusion module (26) in the first second fusion device communicates with the first convolutional unit (25) in the last first fusion device, the hierarchical feature fusion module (26) in the last second fusion device communicates with the edge feature extraction module (21), and the hierarchical feature fusion modules (26) in the remaining second fusion devices communicate with the first convolutional unit (25) in a one-to-one manner. The second convolutional unit (27) is configured to sequentially perform a convolution operation with a kernel size of 3×3, batch normalization, and SiLU activation function processing on its input image to obtain its output image; wherein, the input image of the second convolutional unit (27) in the first second fusion device is the primary fusion feature map; The output image of all the hierarchical feature fusion modules (26) in the second-level feature fusion network is a secondary fusion feature map.

6. The target detection system based on hierarchical feature fusion according to claim 5, characterized in that, The hierarchical feature fusion module (26) is configured to execute a hierarchical feature fusion algorithm comprising the following steps: B1: The following two processes are performed in parallel: The input 1 is convolved to obtain the convolution result of input 1; Perform convolution on input 2 to obtain the convolution result of input 2; B2: The following three processes will be performed in parallel: The input 1 convolution result is mapped using a local attention mechanism to obtain a first attention image, and the input 1 convolution result is also mapped using a global attention mechanism to obtain a second attention image. Then, the first attention image and the second attention image are concatenated by channels to obtain a first concatenated image. The convolution result of input 1 is summed with the convolution result of input 2, and then the summation result is convolved to obtain a combined convolution image; The input 2 convolution result is mapped using a local attention mechanism to obtain a third attention image, and the input 2 convolution result is also mapped using a global attention mechanism to obtain a fourth attention image. Then, the third attention image and the fourth attention image are concatenated by channels to obtain a second concatenated image. B3: The first stitched image, the convolutional image and the second stitched image are stitched together by channels to obtain the final stitched image. Then, the final stitched image is sequentially subjected to convolution processing, reparameterized convolution processing and convolution processing to obtain the output image of the hierarchical feature fusion module (26). Among them, for the hierarchical feature fusion module (26) in the first fusion device, its input 1 is the output image of the first convolutional unit (25) in the first fusion device, which is the same as the hierarchical feature fusion module (26), and its input 2 is a feature image obtained by the second to (m-1)th feature extraction of the backbone network (1); In the second-level feature fusion network, the input 1 of the hierarchical feature fusion module (26) is the output image of the second convolutional unit (27) in the second fusion device, which is the same as the hierarchical feature fusion module (26), and the input 2 is the output image of a certain first convolutional unit (25) or two edge feature maps output by the edge feature extraction module (21).

7. The target detection system based on hierarchical feature fusion according to claim 5 or 6, characterized in that, The edge fusion device is a network structure consisting of (m-1) multi-scale edge feature fusion modules (23) connected in parallel. All the multi-scale edge feature fusion modules (23) communicate with the edge feature extraction module (21). In addition, all the multi-scale edge feature fusion modules (23) also communicate one-to-one with the hierarchical feature fusion module (26) in the first fusion device at the end and all the hierarchical feature fusion modules (26) in the second hierarchical feature fusion network. The splicing unit (24) communicates with all the multi-scale edge feature fusion modules (23) at the same time. The multi-scale edge feature fusion module (23) is configured to execute a dual-input single-output multi-scale edge feature fusion algorithm to transform the two edge feature maps with the primary fusion feature map or one of the secondary fusion feature maps into the final fusion feature map.

8. The target detection system based on hierarchical feature fusion according to claim 7, characterized in that, The multi-scale edge feature fusion module (23) is configured to perform a multi-scale edge feature fusion algorithm comprising the following steps: C1: Perform channel-by-channel stitching of the two edge feature maps with the primary fusion feature map or one of the secondary fusion feature maps to obtain a channel-stitched image; C2: The channel stitched image is sequentially subjected to convolution processing with a kernel size of 1×1, convolution processing with a kernel size of 3×3, and convolution processing with a kernel size of 1×1 to obtain the final fusion feature map.

9. A target detection method based on hierarchical feature fusion, characterized in that, The target detection system based on hierarchical feature fusion according to any one of claims 1-8 includes the following steps: Step 1: Optimize the model parameters of the network formed by the backbone network (1), the efficient hybrid encoder (2) and the decoding module (3) in sequence using the IPIoU loss function; Step 2: The backbone network (1) performs m feature extractions on the image under test in a compound convolutional manner to obtain m feature images. After each feature extraction in the first (m-1) times, the feature images obtained in this time are output and used as the operation objects for the next feature extraction. After the mth feature extraction, the feature images obtained in this time are output. Step 3: The feature image obtained by the first feature extraction of the backbone network (1) is processed by the efficient hybrid encoder (2) to extract edge features to generate two edge feature maps of different scales, and the feature image obtained by the m-th feature extraction of the backbone network (1) is processed by attention-based internal scale feature interaction to obtain an interactive feature map; then the two edge feature maps, the interactive feature map and all feature images obtained by the second to (m-1)-th feature extraction of the backbone network (1) are hierarchically fused to obtain a fused feature map; Step 4: The decoding module (3) sequentially performs uncertainty minimum query selection and multiple decoder layer iterations on the fused feature map to generate corresponding category prediction results and bounding box prediction results for each object of the minimum query selection.

Images