An object detection method combining multi-branch decoupling head and feature enhancement

Abstract

The present disclosure provides a target detection method and device that combines a multi-branch decoupling head and feature enhancement. The method selects the YOLOv5 model as a reference framework to design a target detection network for target detection; the target detection network includes a backbone network, a neck network, and a head module. First, the backbone network uses a convolutional neural network to extract features from the input image to obtain feature maps of three different levels; the neck network performs feature enhancement on the feature maps of the three different levels, and then performs feature fusion to further extract features; the feature enhancement adds a void convolution process to expand the receptive field; the head module uses a three-branch decoupling head to separate the category prediction, prediction box positioning, and confidence calculation in target detection into three independent network branches, performs convolution operations on the extracted features, and outputs the category prediction, prediction box positioning, and confidence calculation results. The use of the present invention can improve the accuracy of small-scale target detection in remote sensing images.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing image imaging, and in particular relates to a target detection method and device combining a multi-branch decoupling head and feature enhancement. Background Art

[0002] Object detection, a core task in computer vision, aims to automatically identify and locate specific objects in images or videos. This technology plays a key role in many research areas, including autonomous driving, video surveillance, facial recognition, medical image analysis, and smart retail. Object detection in remote sensing images is a crucial research area. In remote sensing images, many objects are typically small in size, occupying a small pixel area in high-resolution images. Consequently, their features are often less prominent and easily overwhelmed by background or other interfering information. Furthermore, remote sensing images often cover large geographic areas, resulting in a very sparse and uneven distribution of small-scale objects within the image. This makes it extremely difficult for detection algorithms to capture and identify these objects. Therefore, small-scale object detection is one of the challenges of object detection in remote sensing images.

[0003] Traditional early object detection methods, such as VJ and HOG, can be implemented in three steps: region selection, manual feature extraction, classification, and regression. However, manual feature extraction has significant limitations, consumes a large amount of resources, and suffers from low detection accuracy, especially for small-scale objects.

[0004] Convolutional neural network-based object detection algorithms can be categorized into two-stage and single-stage algorithms. Two-stage detection algorithms, such as R-CNN, Fast-RCNN, and Faster-RCNN, employ a two-step detection process: first, a region proposal network is used to select and generate candidate regions, and then these regions are classified and regressed to obtain detection results. However, the detection process generates a large number of candidate boxes, slowing down detection speed. Furthermore, the detection rate for small-scale objects remains low.

[0005] Single-stage detection algorithms include YOLO, YOLOv2, YOLOv3, and YOLOv4. These algorithms eliminate the need to generate candidate regions and instead directly locate the target and output its category detection information, significantly improving detection speed. YOLOv5 builds on the previous YOLO series by introducing adaptive anchor boxes and a domain-specific positive and negative sample allocation strategy, improving both detection accuracy and speed.

[0006] Several improved methods based on the YOLOv5 framework have been developed for small target detection. FE-YOLO introduces the Feature Enhancement Module (FEM) and the Spatial Perception Module (SCAM). This improves the representation of small target features by incorporating a global attention mechanism and high-level contextual information. It also enhances feature robustness through sparse sampling and adaptive spatial learning using deformable convolutions. FFCA-YOLO proposes three lightweight plug-and-play modules for the neck: the Feature Enhancement Module (FEM), the Feature Fusion Module (FFM), and the Spatial Context Perception Module (SCAM). These modules reduce computational resources while improving small target detection accuracy. However, existing methods still suffer from low detection efficiency, missed small target detection, and false detections.

[0007] In response to the main challenges faced by small target detection mentioned above, namely insufficient feature representation and background confusion, this paper selects YOLOv5 as the benchmark framework, and utilizes the high scalability and high performance of YOLOv5 to propose a high-precision small-scale target detection method in remote sensing images to improve the efficiency of small target detection. Summary of the Invention

[0008] Aiming at the characteristics of small-scale targets such as insufficient feature representation and easy confusion with the background, this paper proposes a target detection method combining a multi-branch decoupling head and feature enhancement to solve the difficulties of target detection algorithms in capturing and identifying small-scale targets and improve the accuracy of small-scale target detection in remote sensing images.

[0009] The solution is implemented as follows:

[0010] A multi-branch decoupling head and feature enhancement target detection method is proposed. The YOLOv5 model is selected as the benchmark framework to design a target detection network for target detection. The target detection network includes a backbone network, a neck network, and a head module. The method includes:

[0011] Step 1: The backbone network uses a convolutional neural network to extract features from the input image and obtain feature maps at three different levels;

[0012] Step 2: The neck network performs feature enhancement on the feature maps of the three different levels and then performs feature fusion to further extract features; the feature enhancement is added with dilated convolution processing to expand the receptive field;

[0013] Step 3: The head module uses a three-branch decoupling head to separate the category prediction, prediction box positioning, and confidence calculation in target detection into three independent network branches. It performs convolution operations on the features extracted in step 2 and outputs the category prediction, prediction box positioning, and confidence calculation results.

[0014] Preferably, the neck network performs feature enhancement using a contextual feature enhancement module (EFB), the EFB module including a shortcut structure network branch and three convolution branches; wherein two convolution branches include a dilated convolution module, and each convolution branch uses two asymmetric convolution operations, with kernel sizes of 1×3 and 3×1 respectively;

[0015] After the input feature map enters the shortcut structure network branch and the three convolution branches for processing respectively, the three convolution branches are connected with the output results of the shortcut structure network branch to obtain the output features.

[0016] Preferably, among the three convolution branches in the EFB module, the first convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, a 3×1 convolution layer, and a 3×3 hole convolution module connected in sequence; the second convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, and a 3×1 convolution layer connected in sequence; the third convolution branch includes a 1×1 convolution layer, a 3×1 convolution layer, a 1×3 convolution layer, and a 3×3 hole convolution module connected in sequence.

[0017] Preferably, the three-branch decoupling head includes a 1×1 convolution layer, and the output result of the 1×1 convolution layer enters three branches; each branch adopts an asymmetric convolution with a kernel of 3×1 and 1×3 respectively.

[0018] Preferably, the method further comprises step 4: training the target detection network constructed in steps 1 to 3, using a minimum distance loss function; the minimum distance loss function is composed of the bounding box similarity term L NWD and border distance term L MPD Weighted acquisition; adjust weighted values ​​through training;

[0019] The bounding box similarity term L NWD The method of obtaining is: using Wasserstein distance to calculate the distribution distance between the predicted box and the true box, limiting the Wasserstein distance to between 0 and 1 by normalization, and obtaining the normalized Wasserstein distance as the bounding box similarity item;

[0020] The border distance item L MPD The method of obtaining is: comparing the minimum distance between the four vertices of the predicted box and the real box IoU is the intersection over union (IoU) of the predicted box and the true box, d1 is the distance between the upper left corner of the predicted box and the true box, d2 is the distance between the lower right corner of the predicted box and the true box, and w and h are the width and height of the input image, respectively.

[0021] The present invention also provides a target detection device combining a multi-branch decoupling head and feature enhancement, the device comprising a target detection network and a training module; wherein,

[0022] The target detection network uses the YOLOv5 model as the benchmark framework, including the backbone network, neck network and head module;

[0023] The backbone network uses a convolutional neural network to extract features from the input image and obtain feature maps at three different levels;

[0024] The neck network performs feature enhancement, feature fusion, and up-construction processing on the feature maps of the three different levels respectively; the feature enhancement module adopts a contextual feature enhancement module (EFB), which includes a shortcut structure network branch and multiple convolution branches; among which, some branches of the multiple convolution branches contain atrous convolution (Atrous Conv) modules, and each convolution branch uses two asymmetric convolution operations; after the input feature map enters the shortcut structure network branch and all convolution branches for processing respectively, each convolution branch is connected with the output result of the shortcut structure network branch to obtain the output feature;

[0025] The head module uses a three-branch decoupling head, which divides the category prediction, prediction box positioning, and confidence calculation in target detection into three independent network branches. It performs convolution operations on the features extracted by the neck network and outputs the category prediction, prediction box positioning, and confidence calculation results.

[0026] The training module is used to train the target detection network.

[0027] Preferably, the EFB module includes a shortcut structure network branch and three convolution branches; two of the convolution branches contain a hole convolution module; the convolution layer of the asymmetric convolution operation on these two convolution branches is designed as follows: one of the convolution branches first performs a 1×3 convolution and then a 3×1 convolution; the other convolution branch first performs a 3×1 convolution and then a 1×3 convolution.

[0028] Preferably, the three-branch decoupling head in the head module includes a 1×1 convolution layer, and the output result of the 1×1 convolution layer enters three branches; each branch adopts asymmetric convolution with kernels of 3×1 and 1×3 respectively.

[0029] Preferably, the training module adopts the minimum distance loss function when training; the minimum distance loss function is composed of the bounding box similarity term L NWD and border distance term L MPD Weighted acquisition; adjust weighted values ​​through training;

[0030] The bounding box similarity term L NWDThe method of obtaining is: using Wasserstein distance to calculate the distribution distance between the predicted box and the true box, limiting the Wasserstein distance to between 0 and 1 by normalization, and obtaining the normalized Wasserstein distance as the bounding box similarity item;

[0031] The border distance item L MPD The method of obtaining is: comparing the minimum distance between the four vertices of the predicted box and the real box IoU is the intersection over union (IoU) of the predicted box and the true box, d1 is the distance between the upper left corner of the predicted box and the true box, d2 is the distance between the lower right corner of the predicted box and the true box, and w and h are the width and height of the input image, respectively.

[0032] Beneficial effects:

[0033] (1) To address the background confusion problem, the present invention proposes a new multi-branch decoupling head. A three-branch decoupling head is adopted to separate the three core tasks of target detection, namely category prediction, prediction box positioning and confidence calculation, into three independent branches. By decoupling the classification and positioning tasks, the model can focus more on positioning targets and spatial information without being disturbed by background features, thereby enhancing the detection capability of small objects.

[0034] (2) To address the problem of insufficient feature representation, this paper proposes a new contextual feature enhancement module (EFB), which aims to capture more contextual information through multi-branch convolution and dilated convolution, which helps to better identify and locate small-scale targets.

[0035] (3) The present invention also proposes a new loss function NWMP-IoU, namely the minimum distance loss function, which measures the similarity of the bounding box by normalizing the Gaussian form of Wasserstein distance, and combines the minimum distance between the four vertices of the bounding box to better reflect the slight position offset and enhance the ability to measure position bias. When facing small-scale target detection, it can more accurately measure the difference between the predicted box and the real box, thereby improving the accuracy of small-scale object detection.

[0036] (4) In a preferred solution, each branch of the three-branch decoupling head adopts asymmetric convolution, which reduces the number of parameters and the amount of computation. At the same time, since convolution can be performed in both horizontal and vertical directions, the receptive field is significantly expanded and the ability to capture contextual information is improved.

[0037] (5) In a preferred solution, the dilated convolution of the EFB module is only used on two branches, thereby reducing the number of parameters and inference time. In addition, each branch uses two asymmetric convolution operations to capture more contextual information, thereby further enhancing the feature information of small-scale objects and improving the recognition accuracy of small-scale objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a schematic diagram of the overall structure of an object detection method combining a multi-branch decoupling head and feature enhancement provided by the present invention;

[0039] Figure 2 A schematic diagram of the structure of the context feature enhancement module provided by the present invention;

[0040] Figure 3 This is a structural schematic diagram of the multi-branch decoupling head module provided by the present invention. DETAILED DESCRIPTION

[0041] In order to improve the accuracy of small-scale target detection, this paper proposes a new target detection scheme that combines a multi-branch decoupling head and contextual feature enhancement to address the difficulties of insufficient target feature representation and background confusion in small-scale target detection.

[0042] To address the problem of insufficient feature representation, the ability to perceive small-scale targets can be effectively enhanced by utilizing local and global contextual information. To this end, this paper proposes a new contextual feature enhancement module EFB (Enhance Feature Block). By adding this contextual feature enhancement module to the neck of the detection model, multi-branch dilated convolution is used to expand the receptive field and improve feature capture capabilities. Multi-branch asymmetric convolution is used to capture more contextual information, which helps to better identify and locate small-scale targets.

[0043] To address the background confusion problem, this paper proposes a new decoupling head structure - the Multi-Decoupled Head (MDH). It adopts a three-branch decoupling head and separates the three core tasks in target detection, namely category prediction, prediction box positioning, and confidence calculation, into three independent branches. Due to the decoupling of classification and positioning tasks, the model can better focus on the spatial information of the target in the positioning task without being interfered by background features, thus avoiding background confusion.

[0044] Furthermore, the present invention also proposes a new loss function NWMP-IoU (Normalized Wasserstein and Minimum Point distance Intersection over Union). Through the design of the loss function, it better reflects small position offsets and enhances the ability to measure position bias. When facing small-scale target detection, it can more accurately measure the difference between the predicted box and the real box, thereby improving the accuracy of small-scale target detection.

[0045] The present invention is described in detail below with reference to the accompanying drawings and embodiments.

[0046] Figure 1 The overall structural diagram of the target detection method combining context feature enhancement and multi-branch decoupling head of the present invention is shown as follows: Figure 1 As shown in the figure, this method selects the YOLOv5 model as the benchmark framework to design the target detection network, which includes a backbone network (Backbone), a neck network (Neck), and a head module (Head). The present invention mainly improves the enhancement module in the neck network and the multi-decoupling head in the head module.

[0047] The method comprises the following steps:

[0048] Step 1: The backbone network extracts multi-level features.

[0049] like Figure 1 As shown in Figure 1, the backbone network processes the input image through multiple layers of convolution and pooling operations, gradually reducing the size of the feature map while increasing the number of channels. After the backbone network's feature extraction operation, three feature maps at different levels are obtained.

[0050] In this embodiment, the backbone network uses a two-stage convolution + C3 module + convolution + C3 module structure to obtain the first-level feature F1. The first-level feature F1 is processed through the convolution + C3 module structure to obtain the second-level feature F2. The second-level feature F2 is processed through the convolution + C3 module + SPPF module structure to obtain the third-level feature F3. F1, F2, and F3 are input to the neck network.

[0051] Step 2: The neck network performs feature enhancement on the feature maps of the three different levels, and then performs feature fusion to further extract features.

[0052] In this step, the feature maps of three different levels obtained by the backbone network are respectively input into the three context feature enhancement modules EFB of the neck network for processing, and then combined with feature fusion and upsampling processing to obtain the output features f1, f2, and f3 of the three channels.

[0053] Small-scale targets contain fewer semantic features due to their small size. At the same time, due to the limited extraction capabilities of the feature extraction network, the receptive field is often small in the shallow layers of the network, making it easy to lose the features of small-scale targets. Therefore, in order to process images with different scales and complex backgrounds, enhance the network's feature expression capabilities, reduce the number of parameters, and improve detection efficiency, this paper introduces the RFB feature enhancement module concept and proposes a new contextual feature enhancement module EFB (Context Feature Block), which aims to improve the ability to capture small-scale target features by expanding the receptive field.

[0054] The EFB module structure is as follows Figure 2As shown, a multi-branch convolutional architecture is employed to capture richer semantic information. The first branch employs a shortcut structure to mitigate the vanishing gradient problem in deep networks, promoting better training and performance. Atrous convolution is employed to expand the receptive field. To reduce the number of parameters and inference time, the contextual feature enhancement module uses atrous convolution in only two branches, improving detection accuracy while reducing inference speed. Each branch employs two asymmetric convolution operations, with kernel sizes of 1×3 and 3×1, respectively, to capture more contextual information. Finally, the three branches are concatenated with the shortcut results to generate the output features.

[0055] like Figure 2 A specific structure shown in FIG. 4 is a module comprising a shortcut structure network branch and three convolution branches; wherein the two convolution branches contain atrous convolution (Atrous Conv) modules, and each convolution branch uses two asymmetric convolution operations. In this embodiment, on the two convolution branches with atrous convolution, the asymmetric convolution operations are designed in the opposite order, that is, the first convolution branch comprises a 1×1 convolution layer, a 1×3 convolution layer, a 3×1 convolution layer, and a 3×3 atrous convolution module connected in sequence; the second convolution branch comprises a 1×1 convolution layer, a 1×3 convolution layer, and a 3×1 convolution layer connected in sequence; the third convolution branch comprises a 1×1 convolution layer, a 3×1 convolution layer, a 1×3 convolution layer, and a 3×3 atrous convolution module connected in sequence.

[0056] After the input feature map (F1 / F2 / F3) enters the shortcut structure network branch and the three convolution branches for processing respectively, the three convolution branches are connected with the output results of the shortcut structure network branch to obtain the output features.

[0057] Step 3: The head module uses a three-branch decoupling head to separate the category prediction, prediction box positioning, and confidence calculation in target detection into three independent network branches. It performs convolution operations on the features extracted in step 2 and outputs the category prediction, prediction box positioning, and confidence calculation results.

[0058] The head module contains three multi-branch decoupled heads, which process the three input features f1, f2, and f3 respectively.

[0059] Based on the two-branch decoupling head proposed by YOLOX, in order to reduce the number of parameters and improve the detection accuracy, the present invention proposes a three-branch decoupling head, the structure of which is as follows: Figure 3As shown in the figure, the target category (Cls.), predicted box position (Reg.), and image confidence (IoU.) are independently convolved into three network branches. By comparing the output with the label during the learning process, the network parameters of each branch of the three-branch decoupling head are updated, and the three branches learn the corresponding targets.

[0060] The three-branch decoupling head consists of a 1×1 convolutional layer whose output is fed into three branches. Each branch employs asymmetric convolution. First, a 1×1 convolution is performed on the input. Then, asymmetric convolutions with 3×1 and 1×3 kernels are applied to each branch, respectively. This reduces the number of parameters and computational complexity. Furthermore, the ability to perform convolution in both the horizontal and vertical directions significantly expands the receptive field and improves the ability to capture contextual information.

[0061] Due to the decoupling of the positioning task and the classification task, the model can focus more on the spatial information of the target when processing the positioning task without being disturbed by background features, thereby reducing the possibility of background confusion and enhancing the ability to detect small objects.

[0062] Step 4: Train the target detection network constructed in steps 1 to 3.

[0063] The training of the present invention adopts the minimum distance loss function, which is composed of the bounding box similarity term L NWD and border distance term L MPD Obtained by weighting; adjust weighted values ​​through training.

[0064] The bounding box similarity term L NWD The acquisition method is as follows: for tiny objects, their foreground pixels are often distributed in the center of the bounding box. Therefore, in order to better capture small-scale targets, different weights are assigned to the pixels in the bounding box. The present invention introduces the idea of ​​NWD (normalized Wasserstein distance) and uses a two-dimensional Gaussian distribution to model the bounding box, where the center pixel of the bounding box has the highest weight, and it decreases from the center to the boundary. The Wasserstein distance is then used to calculate the distribution distance between the predicted box and the true box, and finally the Wasserstein distance is limited to between 0 and 1 by normalization. Assuming that the predicted box and the true box are A = (x1, y1, w1, h1) and B = (x2, y2, w2, h2), respectively, and their two-dimensional Gaussian distributions are N1, N2, w1 and w2 are the widths of the predicted box and the true box, respectively, and h1 and h2 are the heights of the predicted box and the true box, respectively, then the distance metric calculation formula is:

[0065]

[0066] Then normalize its exponent to get the normalized Wasserstein distance:

[0067]

[0068] Where C is a constant that is closely related to the data set.

[0069] The border distance item L MPD The method of obtaining is as follows: when the predicted box and the true box have similar aspect ratios but different sizes, the traditional IoU (Intersection over Union) optimization gradient efficiency is low. Therefore, this paper also introduces the MPDIoU idea to calculate the similarity between boxes by comparing the minimum distance between the four vertices of the bounding box. Assume that the predicted box A and the true box B, the coordinates of the upper left corner and the lower right corner of the predicted box A are (x l1 ,y u1 ), (x r1 ,y d1 ), the coordinates of the upper left corner and lower right corner of the real box B are (x l2 ,y u2 ), (x r2 ,y d2 ), the Euclidean distance between its upper left corner and lower right corner is calculated as follows:

[0070]

[0071] The loss function calculation formula is:

[0072]

[0073] Among them, w and h represent the width and height of the input image, Represents the normalized distance between the upper left and lower right corners of the two groups of points, ∩ represents intersection, ∪ represents the AND operation, Is the intersection of the predicted box A and the real box B. By subtracting the normalized distance between the corresponding points on the basis of IoU, the closer the two boxes are, the closer L MPD The larger the distance, the greater the L MPD In addition, by calculating the distance between the coordinates of the vertices of the bounding box, the shortcomings of gradient optimization under large scale differences can be effectively solved.

[0074] Then the NWMP-IoU calculation formula proposed in the present invention is:

[0075] L=0.5L NWD +0.5L MPD (6)

[0076] Based on the above method, the present invention provides a target detection device combining context feature enhancement and a multi-branch decoupling head, which includes a target detection network and a training module.

[0077] The target detection network uses the YOLOv5 model as the benchmark framework, including the backbone network, neck network and head module;

[0078] The backbone network uses a convolutional neural network to extract features from the image and obtain feature maps at three different levels;

[0079] The neck network performs feature enhancement, feature fusion, and up-construction processing on the feature maps of the three different levels respectively; the feature enhancement module adopts a contextual feature enhancement module (EFB), which includes a shortcut structure network branch and multiple convolution branches; among which, some branches of the multiple convolution branches contain atrous convolution (Atrous Conv) modules, and each convolution branch uses two asymmetric convolution operations; after the input feature map enters the shortcut structure network branch and all convolution branches for processing respectively, each convolution branch is connected with the output result of the shortcut structure network branch to obtain the output feature;

[0080] The head module uses a three-branch decoupling head, which divides the category prediction, prediction box positioning, and confidence calculation in target detection into three independent network branches. It performs convolution operations on the features extracted by the neck network and outputs the category prediction, prediction box positioning, and confidence calculation results.

[0081] The training module is used to train the target detection network.

[0082] In a preferred embodiment, the EFB module includes a shortcut structure network branch and three convolution branches; two of the convolution branches contain a hole convolution module; the convolution layers of the asymmetric convolution operations on these two convolution branches are designed as follows: one of the convolution branches first performs a 1×3 convolution and then a 3×1 convolution; the other convolution branch first performs a 3×1 convolution and then a 1×3 convolution.

[0083] In a preferred embodiment, the three-branch decoupling head in the head module includes a 1×1 convolution layer, and the output result of the 1×1 convolution layer enters three branches; each branch uses an asymmetric convolution with a kernel of 3×1 and 1×3 respectively.

[0084] In a preferred embodiment, the training module adopts the minimum distance loss function when training; the minimum distance loss function is composed of the bounding box similarity term L NWD and border distance term L MPD Weighted acquisition; adjust weighted values ​​through training;

[0085] The bounding box similarity term LNWD The method of obtaining is: using Wasserstein distance to calculate the distribution distance between the predicted box and the true box, limiting the Wasserstein distance to between 0 and 1 by normalization, and obtaining the normalized Wasserstein distance as the bounding box similarity item;

[0086] The border distance item L MPD The method of obtaining is: comparing the minimum distance between the four vertices of the predicted box and the real box IoU is the intersection over union (IoU) of the predicted box and the true box, d1 is the distance between the upper left corner of the predicted box and the true box, d2 is the distance between the lower right corner of the predicted box and the true box, w1 and w2 are the widths of the predicted box and the true box respectively, and h1 and h2 are the heights of the predicted box and the true box respectively.

[0087] The above specific embodiments merely illustrate the design principles of the present invention. The shapes and names of the components described herein may vary and are not limiting. Therefore, those skilled in the art may modify or substitute equivalents for the technical solutions described in the above embodiments. Such modifications and substitutions, without departing from the inventive spirit and technical solutions of the present invention, shall fall within the scope of protection of the present invention.

Claims

1. A target detection method combining a multi-branch decoupling head and feature enhancement, characterized in that: This method selects the YOLOv5 model as the benchmark framework to design the target detection network for target detection; The target detection network includes a backbone network, a neck network and a head module; the method includes: Step 1: The backbone network uses a convolutional neural network to extract features from the input image and obtain feature maps at three different levels; Step 2: The neck network performs feature enhancement on the feature maps of the three different levels and then performs feature fusion to further extract features; the feature enhancement is added with dilated convolution processing to expand the receptive field; Among them, the neck network adopts the context feature enhancement module EFB for feature enhancement, and EFB includes a shortcut structure network branch and three convolution branches; among the three convolution branches in the EFB, the first convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, a 3×1 convolution layer, and a 3×3 hole convolution module connected in sequence; the second convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, and a 3×1 convolution layer connected in sequence; the third convolution branch includes a 1×1 convolution layer, a 3×1 convolution layer, a 1×3 convolution layer, and a 3×3 hole convolution module connected in sequence; after the input feature map enters the shortcut structure network branch and the three convolution branches for processing, the three convolution branches are connected with the output results of the shortcut structure network branch to obtain the output features; Step 3: The head module uses a three-branch decoupling head to separate the category prediction, prediction box positioning, and confidence calculation in object detection into three independent network branches. It performs convolution operations on the features extracted in step 2 and outputs the category prediction, prediction box positioning, and confidence calculation results. The three-branch decoupling head includes a 1×1 convolution layer, and the output of the 1×1 convolution layer enters three branches; each branch uses asymmetric convolution with kernels of 3×1 and 1×3 respectively; Step 4: Train the target detection network constructed in steps 1 to 3, and use the minimum distance loss function; the minimum distance loss function is composed of the bounding box similarity term and border distance items Weighted acquisition; adjust weighted values ​​through training; The bounding box similarity item The method of obtaining is: using Wasserstein distance to calculate the distribution distance between the predicted box and the true box, limiting the Wasserstein distance to between 0 and 1 by normalization, and obtaining the normalized Wasserstein distance as the bounding box similarity item; The border distance item The method of obtaining is: comparing the minimum distance between the four vertices of the predicted box and the real box ; IoU is the intersection-over-union ratio of the predicted box and the real box, d 1 is the distance between the upper left corner of the predicted box and the real box, d 2 is the distance between the lower right corner of the predicted box and the real box, w and h are the width and height of the input image, respectively.

2. An object detection device combining a multi-branch decoupling head and feature enhancement, comprising an object detection network and a training module, characterized in that: The target detection network uses the YOLOv5 model as the benchmark framework, including the backbone network, neck network and head module; The backbone network uses a convolutional neural network to extract features from the input image and obtain feature maps at three different levels; The neck network performs feature enhancement, feature fusion and upsampling on the feature maps of the three different levels respectively; The feature enhancement module adopts a contextual feature enhancement module EFB, which includes a shortcut structure network branch and three convolution branches; among the three convolution branches in the EFB, the first convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, a 3×1 convolution layer, and a 3×3 hole convolution module connected in sequence; the second convolution branch includes a 1×1 convolution layer, a 1×3 convolution layer, and a 3×1 convolution layer connected in sequence; the third convolution branch includes a 1×1 convolution layer, a 3×1 convolution layer, a 1×3 convolution layer, and a 3×3 hole convolution module connected in sequence; after the input feature map enters the shortcut structure network branch and all convolution branches for processing, each convolution branch is connected to the output result of the shortcut structure network branch to obtain the output feature; The head module uses a three-branch decoupling head, which is used to independently separate category prediction, prediction box positioning, and confidence calculation in target detection into three network branches. It performs convolution operations on the features extracted by the neck network and outputs category prediction, prediction box positioning, and confidence calculation results. The three-branch decoupling head includes a 1×1 convolution layer, and the output of the 1×1 convolution layer enters three branches. Each branch uses asymmetric convolution with kernels of 3×1 and 1×3 respectively. The training module is used to train the target detection network; during training, the minimum distance loss function is used; the minimum distance loss function is composed of the bounding box similarity term and border distance items Weighted acquisition; adjust weighted values ​​through training; The bounding box similarity item The method of obtaining is: using Wasserstein distance to calculate the distribution distance between the predicted box and the true box, limiting the Wasserstein distance to between 0 and 1 by normalization, and obtaining the normalized Wasserstein distance as the bounding box similarity item; The border distance item The method of obtaining is: comparing the minimum distance between the four vertices of the predicted box and the real box ; IoU is the intersection-over-union ratio of the predicted box and the real box, d 1 is the distance between the upper left corner of the predicted box and the real box, d 2 is the distance between the lower right corner of the predicted box and the real box, w and h are the width and height of the input image, respectively.

Images