A lightweight dual-branch convolutional neural network for image target detection and its detection method

Abstract

A lightweight dual-branch convolutional neural network for image object detection and its detection method, which solves the lightweight image object detection task by constructing a dual-path network, consists of a dual-path backbone network, a feature pyramid and a lightweight detection head. The core module of the backbone network is an enhanced shuffle module ESB, and an efficient self-attention module ESAM is used to capture the global dependency of features while maintaining very few model parameters and computational complexity. The backbone network adopts a dual-branch network, one branch maintains high resolution, and the other branch uses continuous downsampling to obtain abstract semantic information. In the feature pyramid, ESAM is expanded to ECAM to capture the relationship between features of different resolutions. The detection head uses lightweight depth-separable convolution to predict borders and categories. Experimental results show that this network architecture and detection method achieve the best balance between detection accuracy and implementation efficiency, and become an effective method for solving lightweight image object detection tasks.

Description

Technical Field

[0001] The present invention belongs to the field of image target detection, and in particular relates to a lightweight dual-branch convolutional neural network for image target detection and a detection method thereof. Background Art

[0002] Object detection is a challenging task in computer vision, which has wide applications in autonomous driving and robotics. Object detection aims to detect objects of interest and assign them semantic labels of specific categories. In recent years, with the development of convolutional neural networks, the performance of object detectors has been greatly improved. Typically, object detectors can be divided into two-stage and one-stage. The two-stage detectors are not very efficient due to their multi-stage characteristics. In contrast, one-stage detectors directly predict object bounding boxes and semantic labels. Although they achieve different balances between detection accuracy and speed, their model size and running efficiency are still unacceptable for resource-constrained devices.

[0003] In recent years, single-path networks have become the mainstream architecture of target detectors, which can be roughly divided into two categories: manual design and neural network architecture search. The first category is mainly inspired by classification networks, such as MobileNet-SSD, which combines MobileNet with the detection head of SSD. ThunderNet uses ShuffleNetV2 as the backbone network and replaces the 3×3 depth-separable convolution in ShufflNetV2 with 5×5 depth-separable convolution. Pelee is an efficient variant of DenseNet and reduces the output resolution of the SSD detection head to reduce the amount of computation. Tiny-DSOD introduces depth-separable convolution in both the backbone network and the feature pyramid part. The Tiny-YOLO series makes the model more efficient by reducing the number of convolutional layers or the number of feature pyramid outputs. The second category builds a network through pre-defined network units and search strategies. MnasNet searches for models by optimizing network accuracy and latency. EfficientNet proposes a scaling factor to control the balance between network depth, width, and resolution. MobileDets searches for efficient detection models by optimizing latency on different platforms.

[0004] Since self-attention has a strong ability to capture long-distance dependencies, it has great potential in the field of object detection, while SENet and GENet use global average pooling to collect information and assign different weights to different channels. Although the above methods have made significant progress, they are still limited by the following shortcomings: (1) The single-path architecture adopts an aggressive downsampling strategy, which loses a lot of detailed information that is useful for locating the target. (2) Although self-attention can effectively encode long-distance dependencies, their matrix multiplication is very computationally resource-intensive. On the other hand, global average pooling is used to collect global contextual information, but its spatial dependency modeling ability is insufficient. Therefore, pursuing the optimal balance between detection performance and implementation efficiency remains an open research problem in the lightweight image object detection task, and is also an urgent problem to be solved in the current lightweight network design task. Summary of the invention

[0005] In view of the above-mentioned defects and shortcomings of the prior art, the purpose of the present invention is to propose a lightweight two-branch convolutional neural network and a detection method for image target detection, so as to achieve an optimal balance between model detection accuracy and efficiency through a lightweight convolutional neural network architecture.

[0006] To achieve the above objectives, the specific solution of the present invention is to propose a convolutional neural network for lightweight image object detection, which is composed of three basic modules: backbone network, feature pyramid and detection head. The core units of the backbone network are the enhanced shuffle block (ESB) and the efficient self-attention module (ESAM) (ESAM is a module inside ESB, see Figure 2 (a), (b)), the enhanced shuffle module (ESB) uses deep separable convolution to extract local convolution features, and the efficient self-attention module (ESAM) is embedded in the enhanced shuffle module (ESB) to enhance its long-distance modeling capability while maintaining very few model parameters and computational complexity. In addition, in order to take into account both the details and semantic information of the image, a dual-branch network is adopted, one branch maintains high resolution, and the other branch uses continuous downsampling to obtain abstract semantic information. In the feature pyramid, the efficient self-attention module (ESAM) is expanded to an efficient cross-attention module (ECAM) to capture the relationship between features at different resolutions. A lightweight deep separable convolution is used in the detection head to generate the final bounding box and category predictions to obtain the final image object detection results.

[0007] The beneficial effects achieved by the present invention are:

[0008] The network architecture proposed and applied by the present invention has significant progress compared with the existing high-precision model: the present invention designs an efficient dual-branch network architecture for lightweight image target detection, which provides an efficient method for solving the problem of lightweight target detection. The experimental results show that the designed network architecture achieves the best balance between detection performance and implementation efficiency. The enhanced shuffle module ESB in the backbone network adopts efficient depth-separable convolution and efficient self-attention module ESAM, which can effectively reduce the computational complexity of the network, so that the network can maintain very few model parameters, while enhancing the global modeling ability and expanding the receptive field (receptive field, that is, the area size of each pixel on the feature map output by each layer of the convolutional neural network mapped on the original image), which not only does not increase the computational burden, but also improves the effect of feature extraction. In the feature pyramid, the efficient self-attention module ESAM is expanded to an efficient cross-attention module ECAM to calculate the correlation of cross-scale features. The efficient cross-attention module ECAM includes two forms: ECAM-TD fused from top to bottom and ECAM-BU fused from bottom to top, which can collect more context information and further improve the feature expression ability of the network. In the detection head, lightweight depth-wise separable convolutions are used for prediction, ensuring that the network can achieve the best compromise between accuracy and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a diagram of an efficient dual-branch network architecture for lightweight image target detection in an embodiment of the present invention.

[0010] Figure 2 Schematic diagrams of (a) Bi-FM, (b) ConvBlock, (c) ESB and (d) ESB,s in embodiments of the present invention.

[0011] Figure 3 Schematic diagram of ESAM in an embodiment of the present invention.

[0012] Figure 4 Schematic diagram of a top-down fusion form of ECAM (ECAM-TD) in an embodiment of the present invention, including (a) an ECAM-TD module and (b) a bottleneck block.

[0013] Figure 5 Schematic diagram of the bottom-up fusion form of ECAM (ECAM-BU) in an embodiment of the present invention, including (a) ECAM-BU module and (b) bottleneck block.

[0014] Figure 6 This is a qualitative result diagram of the detection of the neural network on the MS COCO benchmark in an embodiment of the present invention. DETAILED DESCRIPTION

[0015] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0016] from Figure 1 It can be seen that the overall network framework designed by the present invention is an efficient dual-branch network. It consists of three basic parts, including the backbone network, feature pyramid and detection head. Figure 1 The whole framework is called a dual-branch convolutional neural network for lightweight image object detection. The overall features of the network framework designed by the present invention are summarized as follows:

[0017] (1) First, we designed Figure 1 The network structure shown in the figure has a backbone network that is an efficient dual-path architecture, which includes two paths: a high-resolution path and a low-resolution path. The high-resolution path obtains low-level spatial positioning information by maintaining a relatively high resolution (1 / 8 input image resolution); the low-resolution path obtains high-level semantic information by continuously downsampling to obtain a relatively low resolution (1 / 32 input image resolution). Considering the complementarity of the two resolutions, a bidirectional fusion module (Bi-FM, Bi-direction Fusion Module) is adopted to strengthen the information interaction of cross-resolution features of the two paths. In the feature pyramid network, an efficient cross-attention module (ECAM, Efficient Cross-Attention Module) is used to enhance the expression ability of multi-scale features. In the detection head, lightweight deep convolution and point convolution are used for border and category prediction.

[0018] (2) Secondly, the core component of the backbone network is a novel enhanced shuffle block (ESB), which uses lightweight deep convolution and point convolution to extract local features, and uses a lightweight efficient self-attention module (ESAM) to effectively reduce the computational complexity of the network, allowing the network to maintain very few model parameters and improve the feature extraction effect, ensuring that the network can achieve the best compromise between accuracy and efficiency.

[0019] (3) The efficient dual-branch network architecture of the present invention is constructed by combining the backbone network, the feature pyramid network and the detection head. The backbone network is used for feature extraction to obtain low-resolution deep semantic feature maps and high-resolution shallow detail feature maps. The feature pyramid network fuses multiple outputs of the backbone network and feeds them into the detection head, which uses multiple lightweight convolutions to predict categories (C channels, including background categories) and border offsets (4 channels).

[0020] according to Figure 1The overall network framework can efficiently extract the semantic information and detail information of the image and can be trained end-to-end. Compared with the recent mainstream lightweight networks, the designed network architecture achieves the best balance between detection accuracy and implementation efficiency.

[0021] Combination Figure 1 , the method for image target detection using the network designed by the present invention is described, and the steps (Step, hereinafter referred to as S) are as follows:

[0022] S1. Perform a 3×3 convolution with a step size of 2 and a maximum pooling with a step size of 2 on the input original image to obtain the first feature map with a resolution of 1 / 4 of the original image and a channel number of 32.

[0023] S2. Use one ESB,s and three ESB units in turn to extract features from the first feature map to obtain a second feature map with a resolution of 1 / 8 of the original image and a channel number of 128.

[0024] S3. Use one ESB,s and seven ESB units in turn to extract features from the second feature map to obtain a third feature map with a resolution of 1 / 16 of the original image and a channel number of 256.

[0025] S4. Use eight ESB units in sequence to extract features from the second feature map to obtain a fourth feature map with a resolution of 1 / 8 of the original image and a channel number of 128.

[0026] S5. Use the Bi-FM module to fuse the third and fourth feature maps to obtain the fifth and sixth feature maps, with resolutions of 1 / 16 and 1 / 8 of the original image, and the number of channels is 256 and 128 respectively.

[0027] S6. Use one ESB,s and three ESB units in sequence to extract features from the fifth feature map to obtain a seventh feature map with a resolution of 1 / 32 of the original image and a channel number of 512.

[0028] S7. Use four ESB units in sequence to extract features from the sixth feature map to obtain an eighth feature map with a resolution of 1 / 8 of the original image and a channel number of 128.

[0029] S8. Use the Bi-FM module to fuse the seventh and eighth feature maps to obtain the ninth and tenth feature maps, with resolutions of 1 / 32 and 1 / 8 of the original images, and channel numbers of 512 and 128, respectively.

[0030] S9. Use 1×1 convolution on the fifth feature map to obtain an eleventh feature map with a resolution of 1 / 16 of the original image and a channel number of 128.

[0031] S10. Use 1×1 convolution on the ninth feature map to obtain a twelfth feature map with a resolution of 1 / 32 of the original image and a channel number of 128.

[0032] S11. Use 1×1 convolution on the tenth feature map to obtain a thirteenth feature map with a resolution of 1 / 8 of the original image and a channel number of 128.

[0033] S12, sending the eleventh and twelfth feature maps to ECAM-TD to obtain a fourteenth feature map with a resolution of 1 / 16 of the original image and a channel number of 128.

[0034] S13, sending the thirteenth and fourteenth feature maps to ECAM-TD to obtain a fifteenth feature map with a resolution of 1 / 8 of the original image and a channel number of 128.

[0035] S14, sending the fourteenth and fifteenth feature maps to ECAM-BU to obtain a sixteenth feature map with a resolution of 1 / 16 of the original image and a channel number of 128.

[0036] S15, the sixteenth feature map remains unchanged, and the seventeenth feature map with a resolution of 1 / 16 of the original image and a channel number of 128 is obtained.

[0037] S16. Send the eleventh and seventeenth feature maps to ECAM-BU to obtain the seventeenth feature map with a resolution of 1 / 32 of the original image and a channel number of 128.

[0038] S17. Use two cascaded ConvBlocks on the fifteenth feature map to obtain an eighteenth feature map with a resolution of 1 / 8 of the original image and a channel number of 81.

[0039] S18. Use two cascaded ConvBlocks on the fifteenth feature map to obtain a twentieth feature map with a resolution of 1 / 8 of the original image and a channel number of 4.

[0040] S19. Use two cascaded ConvBlocks for the seventeenth feature map to obtain a twenty-first feature map with a resolution of 1 / 16 of the original image and a channel number of 81.

[0041] S20. Use two cascaded ConvBlocks for the seventeenth feature map to obtain a twenty-second feature map with a resolution of 1 / 16 of the original image and a channel number of 4.

[0042] S21, using two cascaded ConvBlocks for the eighteenth feature map to obtain a twenty-third feature map with a resolution of 1 / 32 of the original image and a channel number of 81;

[0043] S22. Use two cascaded ConvBlocks for the eighteenth feature map to obtain a twenty-fourth feature map with a resolution of 1 / 32 of the original image and a channel number of 4.

[0044] S23. Arrange the feature maps obtained in S17-22 to obtain the result of image target detection, including target category and border.

[0045] Special note: The above steps S1-S23 are combined with the attached Figure 1 The overall network structure diagram of the present invention is described, the steps and attached Figure 1 It should be noted that the overall network designed by the present invention is tested on the public benchmark MS COCO for image object detection. Figure 1 Take the MS COCO benchmark as an example.

[0046] Figure 2 The core modules of the entire network are (a) Bi-FM, (b) ConvBlock, (c) ESB, and (d) ESB,s, which are described in detail as follows:

[0047] Figure 2 (a) is the bidirectional feature fusion module (Bi-FM, Bi-direction Fusion Module), which uses 5×5 depth-wise separable convolution with a stride of 2 for the high-resolution features, then uses 1×1 convolution to increase its number of channels to the same as the low-resolution features, and then scales it to the same as the low-resolution features before adding them to them; at the same time, point convolution is used for the low-resolution features to make their channels the same as the high-resolution features, and then bilinear interpolation is used to make their resolution the same as the high-resolution features, and then they are added to them. This module enhances the information interaction of cross-resolution features without increasing the computational burden.

[0048] Figure 2 (b) is the ConvBlock module, which consists of a lightweight 3×3 depth-wise separable convolution and point convolution.

[0049] Figure 2 (c) is an enhanced shuffle block (ESB). ESB separates the input features into two low-dimensional maps of the same channel. One branch passes through 1×1, 5×5 depth-separable convolution with a step size of 1 and 1×1 to extract local convolution features, and then uses an efficient self-attention module (ESAM) to capture long-distance dependencies; the other branch remains unchanged, the outputs of the two branches are spliced, and shuffle units are used for information exchange.

[0050] Figure 2(d) is the enhanced shuffle block with stride (ESB,s). The input features of ESB,s are separated into two low-dimensional maps of the same channel. One branch passes through 1×1 and 5×5 depthwise separable convolutions with a stride of 2 in turn, and the channel remains unchanged. Then, the number of channels is increased by 1×1 convolution, and then the global features are extracted using ESAM; the other branch passes through 5×5 depthwise separable convolution with a stride of 2 for downsampling, and then passes through 1×1 convolution to increase the number of channels by 1 times; the outputs of the two branches are spliced ​​and information exchange is performed using shuffle units.

[0051] Figure 3 The efficient self-attention module (ESAM) designed for the present invention (ESAM is a module inside ESB, see Figure 2 (a), (b)), given an input F∈R C×H×W (C represents the channel of the feature, H and W represent the height and width of the feature), first rearrange its dimensions into a 2D sequence X∈R HW×C , which is used to calculate the spatial attention map S sp and channel attention map S ch In spatial attention, the input sequence X is first fed into a linear mapping to generate a low-dimensional embedding (q, k are used to distinguish these two linear mappings and are used to generate low-dimensional mappings Q′ sp and spatial key feature K sp ), r is a non-negative scaling factor, set to 8. Then, the low-dimensional mapping Q′ sp After global average pooling P g (·) Generate spatial query feature Q sp ∈R 1×C / r : At the same time, X is fed into the linear map To generate the spatial key feature K sp ∈R HW×C / r :

[0052]

[0053] After that, the spatial key feature K sp and the transpose of the spatial query feature Use matrix multiplication to calculate the relationship matrix, which is passed through the sigmoid function σ(·) to generate the spatial attention map S sp ∈R HW×1 :

[0054]

[0055] In channel attention, given two linear mappings and Channel query feature Q ch ∈R HW×1 and channel key feature K ch ∈R HW×C / r With Q sp and K sp The generation method is similar:

[0056]

[0057] After that, the channel relationship graph is obtained by querying the channel features and channel key-value feature K ch The matrix is ​​multiplied between them, and then it is linearly mapped in turn. And sigmoid function to generate channel attention map S ch :

[0058]

[0059] Last S ch and S sp is used to weight the input X:

[0060]

[0061] in, represents element-by-element addition, ⊙ represents element-by-element multiplication, and the output sequence By rearranging, the dimensions become Prepare for the next splicing operation in ESB. ESAM has both high computational efficiency and powerful feature expression capabilities. It uses global pooling and linear mapping Reduced spatial key feature Q sp In addition, the channel reduction factor r is set to 8, which greatly reduces the computational cost of matrix multiplication. Unlike SENet and GENet, which only use global pooling to aggregate features, the matrix multiplication in ESAM is conducive to encoding long-distance dependencies.

[0062] Figure 4 (a) The Efficient Cross-Attention Module (ECAM-TD) designed for top-down fusion in this paper, given a high-resolution input and low resolution input First, F l Upsample to Then F h , F l Rearrange into 2D sequence X l and X h Used to calculate the spatial attention map And channel attention map In spatial attention, input X l First pass Generate a low-dimensional map r is a non-negative integer and is set to 8; then After a global average pooling P g (·), generating spatial query features At the same time X h is fed into a linear map To generate spatial key features

[0063]

[0064] After that, the transposition of the spatial query features and spatial key-value features Use matrix multiplication to calculate the relationship matrix, and generate the spatial attention map through the sigmoid function σ(·)

[0065]

[0066] In channel attention, given a linear mapping and Channel query features Key-value features with channels The generation method is the same as in formula (6) and The generation method is similar:

[0067]

[0068] After this channel relationship diagram Query the transpose of features by channel and channel key-value features The matrix is ​​multiplied between them, and then it is linearly mapped in turn. And sigmoid function to generate channel attention map:

[0069]

[0070] at last and is used to weight the input X h , while X h There are skip connections with the output path to facilitate training:

[0071]

[0072] in, represents element-by-element addition, Represents element-by-element multiplication, output sequence By rearranging, the dimensions become It then passes through a lightweight bottleneck block to improve the output.

[0073] Figure 4 (b) is the bottleneck block, which first undergoes a 1×1 convolution to reduce the number of channels to 1 / 4, then undergoes two depth-wise separable 3×3 convolutions with a stride of 1, and then undergoes a 1×1 convolution to restore the number of channels to be equal to the number of input channels. There is a jump connection between the input and output to facilitate training.

[0074] Figure 5 (a) is the efficient cross-attention module (ECAM-BU) designed for bottom-up fusion in this paper.

[0075] First high-resolution input is downsampled so that its resolution is Same, then F h ,F l Rearrange into 2D sequence Computing spatial attention map And channel attention map Similar to ECAM-TD; finally and is used to weight the input X l , while X l There are also skip connections to the output path to facilitate training. Output sequence By rearranging, the dimensions become Then it goes through a lightweight bottleneck block to improve the output

[0076] Figure 5 (b) is the bottleneck block, and Figure 4 (b) is the same and will not be described again.

[0077] Figure 6This is a detection result diagram of the lightweight dual-branch convolutional neural network for image target detection proposed by the present invention on the MS COCO benchmark. In order to verify the accuracy and implementation efficiency of the network designed by the present invention, the model was trained, evaluated and predicted on the widely used MSCOCO dataset. The MS COCO training set / validation set / test set contains 11K / 5K / 20K images respectively. During training, the number of detection categories is set to 81, i.e., 80 target categories and 1 background category. After training, the detection results output by multiple lightweight networks are qualitatively compared. The model designed by the present invention achieved an AP of 29.0% on the test set, with only a model size of 2.27M and 1.14GFLOPs. The experimental results show that the designed efficient symmetric network achieves the best balance between detection accuracy and implementation efficiency, and its performance has largely surpassed many advanced models. The model designed by the present invention is nearly twice as large as Tiny-DSOD, but its performance is 5.8% higher than it. Although LightDet's GFLOPs is only half of that of the present invention, its performance is 5.0% lower. Although ThunderNet has an AP of 29.0%, it has a GFLOPs of 1.14GFLOPs. 50 Higher than the present invention, but the AP and AP of the present invention 75 They are 0.9% and 0.8% higher than those of the Figure 6 From the detection results, the present invention accurately detects objects of different sizes, such as the bird in the third row and third column and the person in the fourth row and second column. Whether comparing the results quantitatively or qualitatively, it fully demonstrates the superiority of the dual-branch convolutional neural network proposed in the present invention in the lightweight image target detection task.

[0078] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. A lightweight two-branch convolutional neural network for image object detection, characterized in that: The lightweight dual-branch convolutional neural network includes a backbone network, a feature pyramid network and a detection head connected in sequence, wherein: The backbone network uses a dual-branch architecture. One branch maintains high-resolution features, while the other branch uses continuous downsampling operations to obtain low-resolution features, and uses a bidirectional fusion module Bi-FM to fuse cross-resolution features. Feature Pyramid Network, which uses an efficient cross-attention module ECAM to calculate the relationship between features of different resolutions; The detection head uses the ConvBlock module containing lightweight deep convolution and point convolution to perform category and bounding box prediction and output the final image object detection result; The feature pyramid network is provided with an efficient cross attention module ECAM, which includes an efficient cross attention module ECAM-TD during top-down fusion and an efficient cross attention module ECAM-BU during bottom-up fusion, and is provided with a lightweight bottleneck block to improve output; The efficient cross-attention module ECAM-TD in top-down fusion, given a high-resolution input and low resolution input First, F l Upsample to Then F h ,F l Rearrange into 2D sequence X l and X h Used to calculate the spatial attention map And channel attention map In spatial attention, input X l First pass Generate a low-dimensional map r is a non-negative integer and is set to 8; then After a global average pooling P g (·), generating spatial query features At the same time X h is fed into a linear map To generate spatial key-value features After that, the transposition of the spatial query features and spatial key-value features Use matrix multiplication to calculate the relationship matrix, and generate the spatial attention map through the sigmoid function σ(·) In channel attention, given a linear mapping and Channel query features Channel key value characteristics The generation method is the same as in formula (1) and The generation method is similar: After this channel relationship diagram Query the transpose of features by channel and channel key-value features The matrix is ​​multiplied between them, and then it is linearly mapped in turn. And sigmoid function to generate channel attention map: at last and is used to weight the input X h , while X h There are skip connections with the output path to facilitate training: in, represents element-by-element addition, ⊙ represents element-by-element multiplication, and the output sequence By rearranging, the dimensions become It then passes through a lightweight bottleneck block to improve the output.

2. The lightweight two-branch convolutional neural network for image target detection according to claim 1, characterized in that: The backbone network includes a convolutional layer, a maximum pooling layer, an enhanced shuffle module ESB, an enhanced shuffle module with stride ESB,s and a bidirectional fusion module Bi-FM.

3. A lightweight two-branch convolutional neural network for image target detection according to claim 2, characterized in that: The enhanced shuffle module ESB constitutes a low-dimensional mapping that separates the input features into two identical channels. One branch passes through 1×1, a 5×5 depth-separable convolution with a step size of 1 and 1×1 to extract local convolution features, and then uses an efficient self-attention module ESAM to capture long-distance dependencies; the other branch remains unchanged; the outputs of the two branches are spliced, and information interaction is performed using shuffle units.

4. The lightweight two-branch convolutional neural network for image target detection according to claim 2, characterized in that: The enhanced shuffle module ESB,s with stride separates the input features into two low-dimensional maps of the same channel. One branch undergoes 1×1, 5×5 depth-separable convolutions with a stride of 2 in sequence, and the channel remains unchanged. Then, the number of channels is increased by 1×1 convolution, and then the efficient self-attention module ESAM is used to extract global features; the other branch is down-sampled by 5×5 depth-separable convolutions with a stride of 2, and then the number of channels is increased by 1×1 convolution; the outputs of the two branches are spliced, and information exchange is performed using shuffle units.

5. The lightweight two-branch convolutional neural network for image target detection according to claim 2, characterized in that: The bidirectional feature fusion module Bi-FM uses 5×5 depthwise separable convolution with a step size of 2 for the high-resolution feature, and then uses 1×1 convolution to increase its number of channels to the same as the low-resolution feature, and then scales its resolution to the same as the low-resolution feature before adding it to the low-resolution feature; at the same time, point convolution is used for the low-resolution feature to make its channel the same as the high-resolution feature, and then bilinear interpolation is used to make its resolution the same as the high-resolution feature, and then it is added to it.

6. The lightweight dual-branch convolutional neural network for image target detection according to claim 1, characterized in that: The efficient cross-attention module ECAM-BU in bottom-up fusion first inputs high-resolution is downsampled so that its resolution is Same, then F h ,F l Rearrange into 2D sequence Computing spatial attention map And channel attention map Similar to ECAM-TD; finally and is used to weight the input X l , while X l There is also a jump connection with the output path to facilitate training; the output sequence By rearranging, the dimensions become It then passes through a lightweight bottleneck block to improve the output.

7. The lightweight dual-branch convolutional neural network for image target detection according to claim 1, characterized in that: In the bottleneck block, the input undergoes a 1×1 convolution to reduce the number of channels to 1 / 4, then passes through two depth-wise separable 3×3 convolutions with a stride of 1, and then passes through a 1×1 convolution to restore the number of channels to be equal to the number of input channels. There is a jump connection between the input and output to facilitate training.

8. An image target detection method using the lightweight dual-branch convolutional neural network according to any one of claims 1 to 7, characterized in that: The method comprises the following steps: S1, perform 3×3 convolution with a step size of 2 and maximum pooling with a step size of 2 on the input original image to obtain the first feature map with a resolution of 1 / 4 of the original image and a channel number of 32; S2, use one ESB,s and three ESB units to extract features from the first feature map in sequence, and obtain a second feature map with a resolution of 1 / 8 of the original image and a channel number of 128; S3, use one ESB,s and seven ESB units to extract features from the second feature map in sequence, and obtain a third feature map with a resolution of 1 / 16 of the original image and a channel number of 256; S4, using eight ESB units in sequence to extract features from the second feature map to obtain a fourth feature map with a resolution of 1 / 8 of the original image and a channel number of 128; S5. Use the Bi-FM module to fuse the third and fourth feature maps to obtain the fifth and sixth feature maps, whose resolutions are 1 / 16 and 1 / 8 of the original images, and the number of channels are 256 and 128, respectively. S6, using one ESB,s and three ESB units in turn to extract features from the fifth feature map, to obtain a seventh feature map with a resolution of 1 / 32 of the original image and a channel number of 512; S7, using four ESB units in sequence to extract features from the sixth feature map, to obtain an eighth feature map with a resolution of 1 / 8 of the original image and a channel number of 128; S8, using the Bi-FM module to fuse the seventh and eighth feature maps to obtain the ninth and tenth feature maps, with resolutions of 1 / 32 and 1 / 8 of the original image, and channel numbers of 512 and 128, respectively; S9, use 1×1 convolution on the fifth feature map to obtain an eleventh feature map with a resolution of 1 / 16 of the original image and a channel number of 128; S10, using 1×1 convolution on the ninth feature map to obtain a twelfth feature map with a resolution of 1 / 32 of the original image and a channel number of 128; S11, use 1×1 convolution on the tenth feature map to obtain a thirteenth feature map with a resolution of 1 / 8 of the original image and a channel number of 128; S12, sending the eleventh and twelfth feature maps to ECAM-TD to obtain a fourteenth feature map with a resolution of 1 / 16 of the original image and a channel number of 128; S13, sending the thirteenth and fourteenth feature maps to ECAM-TD to obtain a fifteenth feature map with a resolution of 1 / 8 of the original image and a channel number of 128; S14, sending the fourteenth and fifteenth feature maps to ECAM-BU to obtain a sixteenth feature map with a resolution of 1 / 16 of the original image and a channel number of 128; S15, the sixteenth feature map remains unchanged, and a seventeenth feature map with a resolution of 1 / 16 of the original image and a channel number of 128 is obtained; S16, sending the eleventh and seventeenth feature maps to ECAM-BU to obtain the seventeenth feature map with a resolution of 1 / 32 of the original image and a channel number of 128; S17, using two cascaded ConvBlocks for the fifteenth feature map to obtain an eighteenth feature map with a resolution of 1 / 8 of the original image and a channel number of 81; S18, using two cascaded ConvBlocks for the fifteenth feature map to obtain a twentieth feature map with a resolution of 1 / 8 of the original image and a channel number of 4; S19, using two cascaded ConvBlocks for the seventeenth feature map to obtain a twenty-first feature map with a resolution of 1 / 16 of the original image and a channel number of 81; S20, using two cascaded ConvBlocks for the seventeenth feature map to obtain a twenty-second feature map with a resolution of 1 / 16 of the original image and a channel number of 4; S21, using two cascaded ConvBlocks for the eighteenth feature map to obtain a twenty-third feature map with a resolution of 1 / 32 of the original image and a channel number of 81; S22, using two cascaded ConvBlocks for the eighteenth feature map to obtain a twenty-fourth feature map with a resolution of 1 / 32 of the original image and a channel number of 4; S23. Arrange the feature maps obtained in S17-22 to obtain the result of image target detection, including target category and border.

Images