Target detection method combining multi-branch decoupling head and feature enhancement

Abstract

The invention provides a target detection method and device combining a multi-branch decoupling head and feature enhancement. According to the method, a YOLOv5 model is selected as a reference framework to design a target detection network for target detection; the target detection network includes a backbone network, a neck network, and a head module. The method comprises the following steps: firstly, performing feature extraction on an input image by using a convolutional neural network through a backbone network to obtain three feature maps of different levels; the neck network performs feature enhancement on the three feature maps of different levels and then performs feature fusion so as to further extract features; the feature enhancement is added into cavity convolution processing to enlarge a receptive field; the head module adopts a three-branch decoupling head, independently divides category prediction, prediction frame positioning and confidence calculation in target detection into three network branches, performs convolution operation on extracted features, and outputs category prediction, prediction frame positioning and confidence calculation results. By using the method, the small-scale target detection precision of the remote sensing image can be improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image imaging, and in particular relates to a target detection method and device combining a multi-branch decoupling head and feature enhancement. Background Art

[0002] As a core task in the field of computer vision, object detection aims to automatically identify and locate specific objects in images or videos. This technology plays a key role in many research fields, including autonomous driving, video surveillance, face recognition, medical image analysis, and smart retail. Object detection for remote sensing images is a crucial research field. In remote sensing images, many targets usually have small-scale characteristics, that is, the pixel area they occupy in high-resolution images is small, so the feature performance in the image is usually not significant enough and is easily submerged by background or other interference information. In addition, remote sensing images often cover large geographical areas, resulting in very sparse and uneven distribution of small-scale targets in the image, which makes it more difficult for detection algorithms to capture and identify these targets. Therefore, small-scale target detection is one of the difficulties in remote sensing image target detection.

[0003] The implementation process of traditional early target detection methods, such as VJ and HOG, can be divided into three steps: region selection, manual feature extraction, classification and regression. However, manual feature extraction has great limitations, consumes a lot of resources and has low detection accuracy, especially for small-scale targets.

[0004] Object detection algorithms based on convolutional neural networks can be divided into two-stage detection algorithms and single-stage detection algorithms. Two-stage detection algorithms include R-CNN, Fast-RCNN, Faster-RCNN, etc. The detection method is divided into two steps. First, the candidate regions are selected and generated through the region proposal network, and then these regions are classified and regressed to obtain the detection results. However, the disadvantage is that a large number of candidate boxes are generated during the detection process, which reduces the detection speed. At the same time, there is still a problem of low detection rate for small-scale targets.

[0005] Single-stage detection algorithms include YOLO, YOLOv2, YOLOv3, YOLOv4, etc. Their detection methods do not need to generate candidate regions, but directly locate the target and output the target's category detection information, thereby improving the detection speed to a certain extent. YOLOv5 introduces adaptive anchor boxes and domain positive and negative sample allocation strategies based on the previous YOLO series, which not only improves the detection accuracy, but also improves the detection speed.

[0006] There are currently some improved methods based on the YOLOv5 framework for small target detection. FE-YOLO introduces the feature enhancement module FEM and the spatial perception module SCAM, improves the small target feature expression capability by introducing a global attention mechanism and advanced context information, and enhances feature robustness by performing sparse sampling and adaptive spatial learning through deformable convolution. FFCA-YOLO proposes three lightweight plug-and-play modules for the neck, namely the feature enhancement module FEM, the feature fusion module FFM, and the spatial context perception module SCAM, which improve the accuracy of small target detection while reducing computing resources. However, existing methods still have problems such as low detection efficiency, missed detection of small targets, and false detection.

[0007] In view of the main challenges faced by small target detection mentioned above, namely insufficient feature representation and background confusion, this paper selects YOLOv5 as the benchmark framework, and utilizes the high scalability and high performance of YOLOv5 to propose a high-precision small-scale target detection method for remote sensing images, thereby improving the efficiency of small target detection. Summary of the invention

[0008] Aiming at the characteristics of small-scale targets such as insufficient feature representation and easy confusion with the background, this paper proposes a target detection method combining a multi-branch decoupling head and feature enhancement to solve the dilemma of target detection algorithms in capturing and identifying small-scale targets and improve the accuracy of small-scale target detection in remote sensing images.

[0009] The solution is implemented as follows:

[0010] A multi-branch decoupling head and feature enhancement target detection method, selecting the YOLOv5 model as a benchmark framework to design a target detection network for target detection; the target detection network includes a backbone network, a neck network and a head module; the method includes:

[0011] Step 1: The backbone network uses a convolutional neural network to extract features from the input image and obtain feature maps at three different levels;

[0012] Step 2: The neck network performs feature enhancement on the feature maps of the three different levels respectively, and then performs feature fusion to further extract features; the feature enhancement is added with a dilated convolution process to expand the receptive field;

[0013] Step 3: The head module adopts a three-branch decoupling head to separate the category prediction, prediction box positioning and confidence calculation in target detection into three independent network branches, perform convolution operations on the features extracted in step 2, and output the category prediction, prediction box positioning and confidence calculation results.

[0014] Preferably, the neck network uses a context feature enhancement module (EFB) for feature enhancement, and the EFB module includes a shortcut structure network branch and three convolution branches; wherein two convolution branches include a hole convolution module, and each convolution branch uses two asymmetric convolution operations, and the kernel sizes are 1×3 and 3×1 respectively;

[0015] After the input feature map enters the shortcut structure network branch and the three convolution branches for processing respectively, the three convolution branches are connected with the output results of the shortcut structure network branch to obtain the output features.

[0016] Preferably, among the three convolution branches in the EFB module, the first convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, a 3×1 convolution layer, and a 3×3 hole convolution module connected in sequence; the second convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, and a 3×1 convolution layer connected in sequence; the third convolution branch includes a 1×1 convolution layer, a 3×1 convolution layer, a 1×3 convolution layer, and a 3×3 hole convolution module connected in sequence.

[0017] Preferably, the three-branch decoupling head includes a 1×1 convolution layer, and the output result of the 1×1 convolution layer enters three branches; each branch adopts an asymmetric convolution with a kernel of 3×1 and 1×3 respectively.

[0018] Preferably, the method further comprises step 4: training the target detection network constructed in steps 1 to 3, using a minimum distance loss function for the training; the minimum distance loss function is composed of the bounding box similarity term L NWD and border distance term L MPD Weighted acquisition; weighted values ​​are adjusted through training;

[0019] The bounding box similarity term L NWD The method of obtaining is: using Wasserstein distance to calculate the distribution distance between the predicted box and the real box, limiting the Wasserstein distance to between 0 and 1 by normalization, and obtaining the normalized Wasserstein distance as the bounding box similarity item;

[0020] The border distance term L MPD The method of obtaining is: compare the minimum distance between the four vertices of the predicted box and the real box IoU is the intersection over union (IoU) of the predicted box and the true box, d1 is the distance between the upper left corner of the predicted box and the true box, d2 is the distance between the lower right corner of the predicted box and the true box, and w and h are the width and height of the input image, respectively.

[0021] The present invention also provides a target detection device combining a multi-branch decoupling head and feature enhancement, the device comprising a target detection network and a training module; wherein,

[0022] The target detection network uses the YOLOv5 model as the benchmark framework, including the backbone network, neck network and head module;

[0023] The backbone network uses a convolutional neural network to extract features from the input image and obtain feature maps at three different levels;

[0024] The neck network performs feature enhancement, feature fusion and upper material processing on the feature maps of the three different levels respectively; the feature enhancement module adopts a context feature enhancement module (EFB), and the EFB module includes a shortcut structure network branch and multiple convolution branches; among which, some branches of the multiple convolution branches include atrous convolution (Atrous Conv) modules, and each convolution branch uses two asymmetric convolution operations; after the input feature map enters the shortcut structure network branch and all convolution branches for processing respectively, each convolution branch is connected with the output result of the shortcut structure network branch to obtain the output feature;

[0025] The head module adopts a three-branch decoupling head, which divides the category prediction, prediction box positioning and confidence calculation in target detection into three independent network branches, performs convolution operations on the features extracted by the neck network, and outputs the category prediction, prediction box positioning and confidence calculation results;

[0026] The training module is used to train the target detection network.

[0027] Preferably, the EFB module includes a shortcut structure network branch and three convolution branches; two of the convolution branches contain hole convolution modules; the convolution layers of the asymmetric convolution operations on the two convolution branches are designed as follows: one of the convolution branches first performs a 1×3 convolution and then a 3×1 convolution; the other convolution branch first performs a 3×1 convolution and then a 1×3 convolution.

[0028] Preferably, the three-branch decoupling head in the head module includes a 1×1 convolution layer, and the output result of the 1×1 convolution layer enters three branches; each branch adopts an asymmetric convolution with a kernel of 3×1 and 1×3 respectively.

[0029] Preferably, the training module uses a minimum distance loss function when training; the minimum distance loss function is composed of the bounding box similarity term L NWD and border distance term L MPD Weighted acquisition; weighted values ​​are adjusted through training;

[0030] The bounding box similarity term L NWDThe method of obtaining is: using Wasserstein distance to calculate the distribution distance between the predicted box and the real box, limiting the Wasserstein distance to between 0 and 1 by normalization, and obtaining the normalized Wasserstein distance as the bounding box similarity item;

[0031] The border distance term L MPD The method of obtaining is: compare the minimum distance between the four vertices of the predicted box and the real box IoU is the intersection over union (IoU) of the predicted box and the true box, d1 is the distance between the upper left corner of the predicted box and the true box, d2 is the distance between the lower right corner of the predicted box and the true box, and w and h are the width and height of the input image, respectively.

[0032] Beneficial effects:

[0033] (1) To address the background confusion problem, the present invention proposes a new multi-branch decoupling head. A three-branch decoupling head is adopted to separate the three core tasks of target detection, namely category prediction, prediction box positioning and confidence calculation, into three independent branches. By decoupling the classification and positioning tasks, the model can focus more on positioning targets and spatial information without being disturbed by background features, thereby enhancing the detection capability of small objects.

[0034] (2) To address the problem of insufficient feature representation, this paper proposes a new contextual feature enhancement module (EFB), which aims to capture more contextual information through multi-branch convolution and dilated convolution, which helps to better identify and locate small-scale targets.

[0035] (3) The present invention also proposes a new loss function NWMP-IoU, i.e., the minimum distance loss function, which measures the similarity of the bounding boxes by normalizing the Wasserstein distance in Gaussian form and combines the minimum distance between the four vertices of the bounding box to better reflect the slight position offset and enhance the ability to measure position bias. When facing small-scale target detection, it can more accurately measure the difference between the predicted box and the real box, thereby improving the accuracy of small-scale object detection.

[0036] (4) In a preferred embodiment, each branch of the three-branch decoupling head adopts asymmetric convolution, which reduces the number of parameters and the amount of calculation. At the same time, since convolution can be performed in both horizontal and vertical directions, the receptive field is significantly expanded and the ability to capture contextual information is improved.

[0037] (5) In a preferred solution, the dilated convolution of the EFB module is only used on two of the branches, thereby reducing the number of parameters and the inference time. In addition, each branch uses two asymmetric convolution operations to capture more contextual information, thereby further enhancing the feature information of small-scale objects and improving the recognition accuracy of small-scale objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 A schematic diagram of the overall structure of a target detection method combining a multi-branch decoupling head and feature enhancement provided by the present invention;

[0039] Figure 2 A schematic diagram of the structure of the context feature enhancement module provided by the present invention;

[0040] Figure 3 This is a schematic structural diagram of the multi-branch decoupling head module provided by the present invention. DETAILED DESCRIPTION

[0041] Aiming at the difficulties of insufficient target feature representation and background confusion in small-scale target detection, this paper proposes a new target detection scheme combining multi-branch decoupling head and context feature enhancement to improve the accuracy of small-scale target detection.

[0042] To address the problem of insufficient feature representation, the perception of small-scale targets can be effectively enhanced by utilizing local and global context information. To this end, the present invention proposes a new context feature enhancement module EFB (Enhance Feature Block), which is added to the neck of the detection model, uses multi-branch hole convolution to expand the receptive field, improves feature capture capabilities, and captures more context information through multi-branch asymmetric convolution, which helps to better identify and locate small-scale targets.

[0043] To address the background confusion problem, the present invention proposes a new decoupling head structure - Multi-Decoupled Head (MDH). A three-branch decoupling head is adopted to separate the three core tasks in target detection, namely category prediction, prediction box positioning and confidence calculation, into three independent branches. Due to the decoupling of classification and positioning tasks, the model can better focus on the spatial information of the target in the positioning task without being disturbed by background features, thus avoiding background confusion.

[0044] Furthermore, the present invention also proposes a new loss function NWMP-IoU (Normalized Wasserstein and Minimum Point distance Intersection over Union). Through the design of the loss function, it can better reflect the slight position offset and enhance the ability to measure the position bias. When facing small-scale target detection, it can more accurately measure the difference between the predicted box and the real box, thereby improving the accuracy of small-scale target detection.

[0045] The present invention is described in detail below in conjunction with the accompanying drawings and embodiments.

[0046] Figure 1 The overall structural diagram of the target detection method combining context feature enhancement and multi-branch decoupling head of the present invention is shown as follows: Figure 1 As shown, the method selects the YOLOv5 model as the reference framework to design the target detection network, and the target detection network includes a backbone network (Backbone), a neck network (Neck) and a head module (Head). The present invention mainly improves the enhancement module in the neck network and the multi-decoupling head in the head module.

[0047] The method comprises the following steps:

[0048] Step 1: The backbone network extracts multi-level features.

[0049] like Figure 1 As shown in Figure 1, the backbone network processes the input image through multiple layers of convolution and pooling operations, gradually reducing the size of the feature map while increasing the number of channels. After the feature extraction operation of the backbone network, three feature maps of different levels are obtained.

[0050] In this embodiment, the backbone network adopts the structure of two-level convolution + C3 module + convolution + C3 module to obtain the first-level feature F1; the first-level feature F1 is processed by the structure of convolution + C3 module to obtain the second-level feature F2; the second-level feature F2 is processed by the structure of convolution + C3 module + SPPF module to obtain the third-level feature F3. F1, F2, and F3 are input to the neck network.

[0051] Step 2: The neck network performs feature enhancement on the feature maps of the three different levels respectively, and then performs feature fusion to further extract features.

[0052] In this step, the feature maps of three different levels obtained by the backbone network are respectively input into the three context feature enhancement modules EFB of the neck network for processing, and then combined with feature fusion and upsampling processing to obtain the output features f1, f2, f3 of the three channels.

[0053] Small-scale targets contain fewer semantic features due to their small size. At the same time, due to the limited extraction capability of the feature extraction network, the receptive field is often small in the shallow layer of the network, and the features of small-scale targets are easily lost. Therefore, in order to process images with different scales and complex backgrounds, enhance the feature expression capability of the network, reduce the number of parameters, and improve detection efficiency, the present invention introduces the RFB feature enhancement module idea and proposes a new context feature enhancement module EFB (nhence Feature Block), which aims to improve the ability to capture small-scale target features by expanding the receptive field.

[0054] The EFB module structure is as follows Figure 2As shown in the figure, a multi-branch convolution structure is used to obtain richer semantic information. The first branch uses a shortcut structure to alleviate the gradient vanishing problem in the deep network and promote better training and performance. Atrous Conv is used to expand the receptive field. Atrous Conv is used to reduce the number of parameters and the reasoning time. The context feature enhancement module only uses atrous convolution on two of the branches to improve the detection accuracy while reducing the reasoning speed. Each branch uses two asymmetric convolution operations with kernel sizes of 1×3 and 3×1, respectively, to capture more context information. Finally, the three branches are connected with the shortcut results to obtain the output features.

[0055] like Figure 2 A specific structure shown, the module includes a shortcut structure network branch and three convolution branches; wherein two convolution branches contain atrous convolution (Atrous Conv) modules, and each convolution branch uses two asymmetric convolution operations. In this embodiment, on the two convolution branches with atrous convolution, the asymmetric convolution operations are designed in the opposite order, that is, the first convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, a 3×1 convolution layer, and a 3×3 atrous convolution module connected in sequence; the second convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, and a 3×1 convolution layer connected in sequence; the third convolution branch includes a 1×1 convolution layer, a 3×1 convolution layer, a 1×3 convolution layer, and a 3×3 atrous convolution module connected in sequence.

[0056] After the input feature map (F1 / F2 / F3) enters the shortcut structure network branch and the three convolution branches for processing respectively, the three convolution branches are connected with the output results of the shortcut structure network branch to obtain the output features.

[0057] Step 3: The head module adopts a three-branch decoupling head to separate the category prediction, prediction box positioning and confidence calculation in target detection into three independent network branches, perform convolution operations on the features extracted in step 2, and output the category prediction, prediction box positioning and confidence calculation results.

[0058] The head module contains three multi-branch decoupled heads, which process three input features f1, f2, and f3 respectively.

[0059] Based on the double-branch decoupling head proposed by YOLOX, in order to reduce the number of parameters and improve the detection accuracy, the present invention proposes a three-branch decoupling head, whose structure is as follows: Figure 3As shown in the figure, the target category (Cls.), predicted box position (Reg.) and image confidence (IoU.) are independently convolved into three network branches. By comparing the output with the label during the learning process, the network parameters of each branch of the three-branch decoupling head are updated to achieve the learning of the corresponding targets in the three branches.

[0060] The three-branch decoupling head includes a 1×1 convolution layer, and the output of the 1×1 convolution layer enters three branches; each branch uses asymmetric convolution. First, a convolution operation with a kernel of 1×1 is performed on the input, and then asymmetric convolutions with kernels of 3×1 and 1×3 are used in each branch, which reduces the number of parameters and the amount of calculation. At the same time, since convolution can be performed in both horizontal and vertical directions, the receptive field is significantly expanded, and the ability to capture contextual information is improved.

[0061] Due to the decoupling of the positioning task and the classification task, the model can focus more on the spatial information of the target when processing the positioning task without being disturbed by background features, thereby reducing the possibility of background confusion and enhancing the ability to detect small objects.

[0062] Step 4: Train the target detection network constructed in steps 1 to 3.

[0063] The training of the present invention adopts the minimum distance loss function, which is composed of the bounding box similarity term L NWD and border distance term L MPD Obtained by weighting; adjust weighted values ​​through training.

[0064] The bounding box similarity term L NWD The acquisition method is as follows: For tiny objects, their foreground pixels are often distributed in the center of the bounding box. Therefore, in order to better capture small-scale targets, different weights are assigned to the pixels in the bounding box. The present invention introduces the idea of ​​NWD (normalized Wasserstein distance) and uses a two-dimensional Gaussian distribution to model the bounding box, where the center pixel weight of the bounding box is the highest, and it decreases from the center to the boundary. Then the Wasserstein distance is used to calculate the distribution distance between the predicted box and the true box, and finally the Wasserstein distance is normalized to be limited between 0 and 1. Assuming that the predicted box and the true box are A = (x1, y1, w1, h1) and B = (x2, y2, w2, h2), their two-dimensional Gaussian distributions are N1, N2, w1 and w2 are the widths of the predicted box and the true box, respectively, and h1 and h2 are the heights of the predicted box and the true box, respectively, then the distance metric calculation formula is:

[0065]

[0066] Then normalize its exponent to get the normalized Wasserstein distance:

[0067]

[0068] Where C is a constant that is closely related to the data set.

[0069] The border distance term L MPD The method of obtaining is: when the predicted box and the real box have similar aspect ratios but different sizes, the traditional IoU (Intersection over Union) optimization gradient efficiency is low. Therefore, the present invention also introduces the MPDIoU idea to calculate the similarity between boxes by comparing the minimum distance between the four vertices of the bounding box. Assume that the predicted box A and the real box B, the coordinates of the upper left corner and the lower right corner of the predicted box A are (x l1 ,y u1 ), (x r1 ,y d1 ), the coordinates of the upper left corner and lower right corner of the real box B are (x l2 ,y u2 ), (x r2 ,y d2 ), the Euclidean distance between its upper left corner and lower right corner is calculated as follows:

[0070]

[0071] The loss function calculation formula is:

[0072]

[0073] Among them, w and h represent the width and height of the input image, represents the normalized distance between the two sets of points in the upper left corner and the lower right corner, ∩ represents intersection, ∪ represents the AND operation, It is the intersection-over-union (IoU) of the predicted box A and the real box B. By subtracting the normalized distance between the corresponding points from the IoU, the closer the two boxes are, the smaller the distance between the two boxes will be. MPD The larger the distance, the larger the L MPD In addition, by calculating the distance between the coordinates of the vertices of the bounding box, the shortcomings of gradient optimization under large-scale differences can be effectively solved.

[0074] Then the NWMP-IoU calculation formula proposed by the present invention is:

[0075] L=0.5L NWD +0.5L MPD (6)

[0076] Based on the above method, the present invention provides a target detection device combining context feature enhancement and a multi-branch decoupling head, which includes a target detection network and a training module.

[0077] The target detection network uses the YOLOv5 model as the benchmark framework, including the backbone network, neck network and head module;

[0078] The backbone network uses a convolutional neural network to extract features from images and obtain feature maps at three different levels;

[0079] The neck network performs feature enhancement, feature fusion and upper material processing on the feature maps of the three different levels respectively; the feature enhancement module adopts a context feature enhancement module (EFB), and the EFB module includes a shortcut structure network branch and multiple convolution branches; among which, some branches of the multiple convolution branches include atrous convolution (Atrous Conv) modules, and each convolution branch uses two asymmetric convolution operations; after the input feature map enters the shortcut structure network branch and all convolution branches for processing respectively, each convolution branch is connected with the output result of the shortcut structure network branch to obtain the output feature;

[0080] The head module adopts a three-branch decoupling head, which divides the category prediction, prediction box positioning and confidence calculation in target detection into three independent network branches, performs convolution operations on the features extracted by the neck network, and outputs the category prediction, prediction box positioning and confidence calculation results;

[0081] The training module is used to train the target detection network.

[0082] In a preferred embodiment, the EFB module includes a shortcut structure network branch and three convolution branches; two of the convolution branches contain hole convolution modules; the convolution layers of the asymmetric convolution operations on the two convolution branches are designed as follows: one of the convolution branches first performs a 1×3 convolution and then a 3×1 convolution; the other convolution branch first performs a 3×1 convolution and then a 1×3 convolution.

[0083] In a preferred embodiment, the three-branch decoupling head in the head module includes a 1×1 convolution layer, and the output result of the 1×1 convolution layer enters three branches; each branch uses an asymmetric convolution with a kernel of 3×1 and 1×3, respectively.

[0084] In a preferred embodiment, the training module uses a minimum distance loss function when training; the minimum distance loss function is composed of the bounding box similarity term L NWD and border distance term L MPD Weighted acquisition; weighted values ​​are adjusted through training;

[0085] The bounding box similarity term LNWD The method of obtaining is: using Wasserstein distance to calculate the distribution distance between the predicted box and the real box, limiting the Wasserstein distance to between 0 and 1 by normalization, and obtaining the normalized Wasserstein distance as the bounding box similarity item;

[0086] The border distance term L MPD The method of obtaining is: compare the minimum distance between the four vertices of the predicted box and the real box IoU is the intersection over union (IoU) of the predicted box and the true box, d1 is the distance between the upper left corner of the predicted box and the true box, d2 is the distance between the lower right corner of the predicted box and the true box, w1 and w2 are the widths of the predicted box and the true box, respectively, and h1 and h2 are the heights of the predicted box and the true box, respectively.

[0087] The above specific embodiments only describe the design principle of the present invention. The shapes and names of the components in the description may be different and are not limited. Therefore, those skilled in the art in the field of the present invention may modify or replace the technical solutions recorded in the above embodiments; and these modifications and replacements do not deviate from the creative purpose and technical solutions of the present invention and should all fall within the protection scope of the present invention.

Claims

1. A target detection method combining a multi-branch decoupling head and feature enhancement, characterized in that: This method selects the YOLOv5 model as the benchmark framework to design the target detection network for target detection; The target detection network includes a backbone network, a neck network and a head module; the method includes: Step 1: The backbone network uses a convolutional neural network to extract features from the input image and obtain feature maps at three different levels; Step 2: The neck network performs feature enhancement on the feature maps of the three different levels respectively, and then performs feature fusion to further extract features; the feature enhancement is added with a dilated convolution process to expand the receptive field; Step 3: The head module adopts a three-branch decoupling head to separate the category prediction, prediction box positioning and confidence calculation in target detection into three independent network branches, perform convolution operations on the features extracted in step 2, and output the category prediction, prediction box positioning and confidence calculation results.

2. The method according to claim 1, characterized in that The neck network uses a contextual feature enhancement module (EFB) for feature enhancement. The EFB module includes a shortcut structure network branch and three convolution branches. Two convolution branches contain dilated convolution modules. Each convolution branch uses two asymmetric convolution operations, and the kernel sizes are 1×3 and 3×1 respectively. After the input feature map enters the shortcut structure network branch and the three convolution branches for processing respectively, the three convolution branches are connected with the output results of the shortcut structure network branch to obtain the output features.

3. The method according to claim 1, characterized in that Among the three convolution branches in the EFB module, the first convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, a 3×1 convolution layer, and a 3×3 hole convolution module connected in sequence; the second convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, and a 3×1 convolution layer connected in sequence; the third convolution branch includes a 1×1 convolution layer, a 3×1 convolution layer, a 1×3 convolution layer, and a 3×3 hole convolution module connected in sequence.

4. The method according to claim 1, characterized in that The three-branch decoupling head includes a 1×1 convolution layer, and the output result of the 1×1 convolution layer enters three branches; each branch uses an asymmetric convolution with a kernel of 3×1 and 1×3 respectively.

5. The method according to claim 1, characterized in that The method further comprises step 4: training the target detection network constructed in step 1 to step 3, and the training adopts a minimum distance loss function; the minimum distance loss function is composed of the bounding box similarity term L NWD and border distance term L MPD Weighted acquisition; adjusting weighted values ​​through training; The bounding box similarity term L NWD The method of obtaining is: using Wasserstein distance to calculate the distribution distance between the predicted box and the real box, limiting the Wasserstein distance to between 0 and 1 by normalization, and obtaining the normalized Wasserstein distance as the bounding box similarity item; The border distance term L MPD The method of obtaining is: compare the minimum distance between the four vertices of the predicted box and the real box IoU is the intersection over union (IoU) of the predicted box and the true box, d1 is the distance between the upper left corner of the predicted box and the true box, d2 is the distance between the lower right corner of the predicted box and the true box, and w and h are the width and height of the input image, respectively.

6. An object detection device combining a multi-branch decoupling head and feature enhancement, the device comprising an object detection network and a training module, characterized in that: The target detection network uses the YOLOv5 model as the benchmark framework, including the backbone network, neck network and head module; The backbone network uses a convolutional neural network to extract features from the input image and obtain feature maps at three different levels; The neck network performs feature enhancement, feature fusion and material processing on the feature maps of the three different levels respectively; The feature enhancement module adopts a context feature enhancement module (EFB). The EFB module includes a shortcut structure network branch and multiple convolution branches. Some of the multiple convolution branches contain atrous convolution (Atrous Conv) modules, and each convolution branch uses two asymmetric convolution operations. After the input feature map enters the shortcut structure network branch and all convolution branches for processing, each convolution branch is connected with the output result of the shortcut structure network branch to obtain the output feature. The head module adopts a three-branch decoupling head, which divides the category prediction, prediction box positioning and confidence calculation in target detection into three independent network branches, performs convolution operations on the features extracted by the neck network, and outputs the category prediction, prediction box positioning and confidence calculation results; The training module is used to train the target detection network.

7. The device according to claim 6, characterized in that The EFB module includes a shortcut structure network branch and three convolution branches; two of the convolution branches contain hole convolution modules; the convolution layers of the asymmetric convolution operations on the two convolution branches are designed as follows: one of the convolution branches first performs a 1×3 convolution and then a 3×1 convolution; the other convolution branch first performs a 3×1 convolution and then a 1×3 convolution.

8. The device according to claim 6, characterized in that The three-branch decoupling head in the head module includes a 1×1 convolution layer, and the output result of the 1×1 convolution layer enters three branches; each branch uses an asymmetric convolution with a kernel of 3×1 and 1×3 respectively.

9. The device according to claim 6, characterized in that The training module is trained using the minimum distance loss function; the minimum distance loss function is composed of the bounding box similarity term L NWD and border distance term L MPD Weighted acquisition; weighted values ​​are adjusted through training; The bounding box similarity term L NWD The method of obtaining is: using Wasserstein distance to calculate the distribution distance between the predicted box and the real box, limiting the Wasserstein distance to between 0 and 1 by normalization, and obtaining the normalized Wasserstein distance as the bounding box similarity item; The border distance term L MPD The method of obtaining is: compare the minimum distance between the four vertices of the predicted box and the real box IoU is the intersection over union (IoU) of the predicted box and the true box, d1 is the distance between the upper left corner of the predicted box and the true box, d2 is the distance between the lower right corner of the predicted box and the true box, and w and h are the width and height of the input image, respectively.

Images