Method and device for detecting small targets in traffic scene

Abstract

The invention provides a method and a device for detecting small targets in a traffic scene. The method comprises the following steps: extracting a feature map from a traffic scene image to be detected through a backbone network of a target detection model; fusing the feature maps through a first fusion branch from a deep layer to a shallow layer and a second fusion branch from the shallow layer to the deep layer in a neck network of the target detection model, and selecting two layers of feature maps with the deepest backbone network to be fused with the feature map with the shallowest layer through the first fusion branch; on the basis of the first fusion branch, the secondary shallow layer feature map of the backbone network and the final fusion result of the first fusion branch are fused through a second fusion branch and then fused with the deepest layer feature map of the backbone network; and inputting a final fusion result of the feature fusion method of the target detection model into a head network of the target detection model, and detecting a target from the to-be-detected traffic scene image by the head network. More effective feature fusion is realized, and the detection precision of the small target in the traffic scene is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of small target detection, and in particular to a method and device for detecting small targets in traffic scenes. Background Art

[0002] Small target detection has always been a difficult problem in the field of target detection, because small targets contain less information and are greatly affected by environmental noise. Although there have been many studies on small target detection, these studies mainly focus on dense small target detection and how to improve the ability to extract small target features. However, in traffic scenes, the detection of small traffic targets such as traffic signs and traffic lights is still a major problem. There are two main challenges in small traffic target detection: one is the need to accurately distinguish the category information contained in small targets; the other is how to accurately identify targets in complex environmental noise.

[0003] Although existing detection algorithms have made progress in conventional target detection, they are not sufficient to meet the specific needs of traffic scenarios. Therefore, the applicability of these algorithms is limited, and the performance effect is difficult to achieve as expected. For small target detection in traffic scenarios, most methods choose target detection models based on convolutional neural networks for recognition. However, the feature fusion part of the model is often only sequentially connected, and there is feature redundancy or noise information that affects the detection performance, which is not suitable for small target detection tasks.

[0004] Therefore, how to design a targeted and efficient feature fusion network for small target detection in traffic scenes is the key to solving the above problems. It has very important research significance and application value for improving traffic safety, realizing intelligent traffic management and autonomous driving. Summary of the invention

[0005] The present invention provides a method and device for detecting small targets in traffic scenes, so as to solve the defects that the detection algorithm in the prior art is not suitable for detecting small targets in traffic scenes and has low detection performance, realize more effective feature fusion, and improve the accuracy of small target detection in traffic scenes.

[0006] The present invention provides a method for detecting small targets in a traffic scene, comprising:

[0007] Extract feature maps from the traffic scene image to be detected through the backbone network of the target detection model;

[0008] The feature maps are fused through the first fusion branch from deep to shallow layers and the second fusion branch from shallow to deep layers in the neck network of the target detection model, and the two deepest feature maps of the backbone network are selected to be fused with the shallowest feature map through the first fusion branch; the sub-shallow feature map of the backbone network is selected to be fused with the fusion result of the first fusion branch through the second fusion branch, and then fused with the deepest feature map of the backbone network;

[0009] The final fusion result of the feature fusion method used by the target detection model is input into the head network of the target detection model, and the head network detects the target from the traffic scene image to be detected.

[0010] According to a method for detecting small targets in traffic scenes provided by the present invention, before fusing the feature map through a first fusion branch from a deep layer to a shallow layer and a second fusion branch from a shallow layer to a deep layer in a neck network of the target detection model, the method further includes:

[0011] The similarity between the fusion result of each current layer in the neck network and the fusion result of the previous layer of the current layer is calculated respectively, the feature layer in the first fusion branch of the neck network is connected one-to-one with the feature layer in the trunk network of the same depth, and the feature layer in the second fusion branch is connected one-to-one with the feature layer in the first fusion branch of the same depth;

[0012] The current layer whose similarity is greater than a preset threshold is deleted.

[0013] According to a small target detection method in a traffic scene provided by the present invention, the feature map is fused through a first fusion branch from a deep layer to a shallow layer in a neck network of the target detection model, including:

[0014] The first fusion branch uses a bilinear interpolation method to upsample the feature map and then fuse it.

[0015] According to a method for detecting small targets in a traffic scene provided by the present invention, a final fusion result of the feature fusion method used by the target detection model is input into a head network of the target detection model, and the head network detects the target from the traffic scene image to be detected, comprising:

[0016] The fusion results of different depths of the feature fusion method used by the target detection model are respectively input into the corresponding detection heads in the head network, and a convolution operation is performed through two network branches in the detection head to obtain the classification result of the target output by one network branch of each detection head and the positioning result of the target output by another network branch;

[0017] Obtaining a final classification result of the target according to the classification result of the target output by the detection head;

[0018] The final positioning result of the target is obtained according to the positioning result of the target output by the detection head.

[0019] According to a method for detecting small targets in a traffic scene provided by the present invention, before extracting a feature map from a traffic scene image to be detected through a backbone network of a target detection model, the method further includes:

[0020] Inputting a traffic scene image sample into the backbone network of the target detection model, and extracting a feature map from the traffic scene image sample;

[0021] fusing the feature graph of the traffic scene image sample through a first fusion branch and a second fusion branch in a neck network of the target detection model;

[0022] Inputting the final fusion result of the feature maps of different depths of the traffic scene image sample by the feature fusion method into the corresponding detection head in the head network, and obtaining the classification result of the target in the traffic scene image sample output by one network branch in each detection head and the positioning result of the target in the traffic scene image sample output by another network branch;

[0023] Calculating the classification loss, the confidence loss of the positioning result, and the bounding box regression loss of the classification result of the target in the traffic scene image sample outputted by the two network branches in each detection head;

[0024] Determine the loss function of the target detection model according to the classification loss of the classification result of the target in the traffic scene image sample outputted by the two network branches in each detection head, the confidence loss of the positioning result and the bounding box regression loss;

[0025] The target detection model is trained according to the loss function.

[0026] According to a small target detection method in a traffic scene provided by the present invention, a bounding box regression loss of a positioning result of a target in an image sample of the traffic scene is calculated, comprising:

[0027] The CIoU loss function is used to calculate the bounding box regression loss of the positioning result of the target.

[0028] The present invention also provides a small target detection device in a traffic scene, comprising:

[0029] An extraction module, used for extracting a feature map from a traffic scene image to be detected through a backbone network of a target detection model;

[0030] A fusion module is used to fuse the feature maps through a first fusion branch from deep to shallow layers and a second fusion branch from shallow to deep layers in the neck network of the target detection model, select the deepest two layers of feature maps of the backbone network and the shallowest feature map to be fused through the first fusion branch; select the secondary shallow feature map of the backbone network and the final fusion result of the first fusion branch to be fused through the second fusion branch, and then fuse with the deepest feature map of the backbone network;

[0031] The detection module is used to input the final fusion result of the feature fusion method used by the target detection model into the head network of the target detection model, and the head network detects the target from the traffic scene image to be detected.

[0032] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, a method for detecting small targets in a traffic scene as described above is implemented.

[0033] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for detecting small targets in a traffic scene as described in any one of the above is implemented.

[0034] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements any of the above-mentioned methods for detecting small targets in traffic scenes.

[0035] The method and device for detecting small targets in traffic scenes provided by the present invention propose a method for detecting small targets in traffic scenes based on an efficient feature fusion network. On the one hand, layers are selected from two fusion branches in the neck network for feature map fusion, thereby reducing information redundancy; on the other hand, the second fusion branch in the neck network can fuse the feature maps generated by the backbone network across the first fusion branch, strengthen the interaction of information flows between different levels, and significantly improve feature expression, thereby achieving more effective feature fusion and improving the detection accuracy of small targets in traffic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 It is a flow chart of a small target detection method in a traffic scene provided by the present invention;

[0038] Figure 2 It is a structural schematic diagram of a target detection model in a small target detection method in a traffic scene provided by the present invention;

[0039] Figure 3 It is a schematic diagram of the structure of the head network in the small target detection method in the traffic scene provided by the present invention;

[0040] Figure 4 It is a visualized heat map of the target detection model and the YOLOv5s model in the small target detection method in the traffic scene provided by the present invention;

[0041] Figure 5 It is a structural schematic diagram of a small target detection device in a traffic scene provided by the present invention;

[0042] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] Combine the following Figure 1 A method for detecting small objects in a traffic scene of the present invention is described, comprising:

[0045] Step 101, extracting a feature map from a traffic scene image to be detected through a backbone network of a target detection model;

[0046] Step 102, the feature maps are fused through the first fusion branch from deep to shallow layers and the second fusion branch from shallow to deep layers in the neck network of the target detection model, the deepest two layers of feature maps of the backbone network are selected to be fused with the shallowest feature map through the first fusion branch; the sub-shallow feature map of the backbone network is selected to be fused with the final fusion result of the first fusion branch through the second fusion branch, and then fused with the deepest feature map of the backbone network;

[0047] Step 103, inputting the final fusion result of the feature fusion method used by the target detection model into the head network of the target detection model, and the head network detects the target from the traffic scene image to be detected.

[0048] As the backbone network deepens, the relationship between local information and global information gradually weakens. Shallow features in the network have higher resolution and can capture more details and location information, but due to weak extraction capabilities, they lack semantic information and may contain more noise; while features in the deep layers of the network contain richer semantic information, but due to lower resolution, they will lack more detailed features. Therefore, fusing these two types of information can significantly improve the detection performance of the target detection model, especially for the detection of small targets.

[0049] The traditional feature pyramid network consists of a single fusion path from deep to shallow layers, which limits the interaction of more information flows. In order to maximize the use of features, YOLOv5s, in addition to the process from deep to shallow layers, introduces a branch from shallow to deep layers in combination with the path aggregation network. This design is conducive to transmitting deep semantic information to the front layer, while integrating shallow detailed information into the deep layer to enhance feature representation capabilities. However, the design of this structure has redundancy problems in terms of parameter quantity and computational complexity, and the fusion of information lacks pertinence, and is just a single feature superposition.

[0050] To address these issues, this embodiment further improves the structure of the path aggregation network and proposes an efficient feature fusion network. Figure 2 As shown. The blue rectangles in the first column represent the backbone network, the blue rectangles in the second and third columns represent the neck network, and the gray rectangles in the fourth column represent the head network. The second column is the first fusion branch from deep to shallow layers in the neck network, and the third column is the second fusion branch from shallow to deep layers in the neck network.

[0051] In the backbone network, feature extraction and downsampling are performed layer by layer on the traffic scene image to be detected. For example, the first layer obtains a 4-fold downsampled feature map, the second layer obtains an 8-fold downsampled feature map, the third layer obtains a 16-fold downsampled feature map, and the fourth layer obtains a 32-fold downsampled feature map.

[0052] Figure 2 In the first fusion branch from the deep layer to the shallow layer of the neck network, the two deepest feature maps are specially selected to be upsampled and fused with the shallowest four-fold downsampled feature map, while the fusion of the second layer feature map is eliminated. The reason for this choice is that the shallower feature layers contain more key details required for detecting small targets and locating objects, while the deepest features provide rich semantic information and help instance recognition. In addition, by selecting some feature maps from the deep and shallow layers for fusion, the redundant and noise information is reduced, and the accuracy of small target detection is further improved.

[0053] For the second fusion branch from shallow to deep layers, the lack of spatial information in deep features is considered to be the reason for the poor small target detection effect. Therefore, the 8x downsampled feature map is directly fused with the previously fused 4x downsampled feature map through the second fusion branch, and then fused with the 32x downsampled feature map. Finally, the feature maps output from the deepest and shallowest layers in the neck network are sent to the head network for detection analysis.

[0054] This embodiment proposes a small target detection method for traffic scenes based on an efficient feature fusion network. On the one hand, layers are selected from the two fusion branches in the neck network for feature map fusion, thereby reducing information redundancy; on the other hand, the second fusion branch in the neck network can fuse the feature maps generated by the backbone network across the first fusion branch, strengthen the interaction of information flows between different levels, and significantly improve feature expression, thereby achieving more effective feature fusion and improving the detection accuracy of small targets in traffic scenes.

[0055] On the basis of the above embodiment, before fusing the feature map through the first fusion branch from the deep layer to the shallow layer and the second fusion branch from the shallow layer to the deep layer in the neck network of the target detection model, the present embodiment further includes:

[0056] The similarity between the fusion result of each current layer in the neck network and the fusion result of the previous layer of the current layer is calculated respectively, the feature layer in the first fusion branch of the neck network is connected one-to-one with the feature layer in the trunk network of the same depth, and the feature layer in the second fusion branch is connected one-to-one with the feature layer in the first fusion branch of the same depth;

[0057] The current layer whose similarity is greater than a preset threshold is deleted.

[0058] A similarity algorithm is used to calculate the similarity between the fusion result of each layer in the neck network and the fusion result of the previous layer of each layer. The similarity algorithm may be cosine distance, Euclidean distance, or Pearson similarity, etc. This embodiment does not limit the similarity algorithm.

[0059] If the similarity between the fusion result of the current layer and the fusion result of the previous layer is greater than the preset threshold, such as 85%, it means that there is a lot of information redundancy between the fusion results of the two layers. The current layer is deleted to reduce the redundancy, which has little effect on the fusion quality.

[0060] On the basis of the above embodiment, in this embodiment, the feature map is fused through the first fusion branch from the deep layer to the shallow layer in the neck network of the target detection model, including:

[0061] The first fusion branch uses a bilinear interpolation method to upsample the feature map and then fuse it.

[0062] In this embodiment, the bilinear interpolation method is used in the first fusion branch of the neck network to replace the original nearest neighbor interpolation upsampling method to upsample the feature map, so that the transition of the feature map is more natural. The calculation process of the bilinear interpolation method is as follows:

[0063]

[0064]

[0065] Among them, A 11 , A 21 , A 12 and A 22 It is a known coordinate point, and the coordinates (x, y) of point P need to be calculated. , , , Bilinear interpolation can be decomposed into two single linear interpolations. First, the two formulas in the curly brackets can determine the interpolation value corresponding to the horizontal coordinate direction of point P; then, the second formula is used to calculate the interpolation value corresponding to the vertical coordinate direction of point P.

[0066] On the basis of the above embodiment, in this embodiment, the final fusion result of the feature fusion method used by the target detection model is input into the head network of the target detection model, and the head network detects the target from the traffic scene image to be detected, including:

[0067] The fusion results of different depths of the feature fusion method used by the target detection model are respectively input into the corresponding detection heads in the head network, and a convolution operation is performed through two network branches in the detection head to obtain the classification result of the target output by one network branch of each detection head and the positioning result of the target output by another network branch;

[0068] Obtaining a final classification result of the target according to the classification result of the target output by the detection head;

[0069] The final positioning result of the target is obtained according to the positioning result of the target output by the detection head.

[0070] In object detection algorithms, there is often a spatial misalignment problem between classification and localization tasks. This means that the two tasks focus on different regions of interest. Specifically, the classification task is more concerned with determining the relationship between the extracted features and the object, while the localization task emphasizes correcting the coordinates of the predicted box based on the actual bounding box. Therefore, using a single convolution operation to handle both tasks simultaneously does not yield the best results.

[0071] In this regard, this embodiment changes the coupled detection head network in the target detection model into a decoupled detection head network, whose structure is as follows: Figure 3 shown.

[0072] The decoupled structure in the head network consists of two independent network branches to learn the classification and localization features of the target. Using this structure not only improves the generalization performance of the model, but also improves the convergence speed during training.

[0073] In addition, comparative analysis and experimental data verified that the top-level and bottom-level features of the neck network contain the richest information. In order to reduce the parameter redundancy of the detection head network, the number of detection heads of the model was reduced to two.

[0074] The classification results of the target output by the two detection heads can be averaged to obtain the final classification result of the target.

[0075] The positioning results of the target output by the two detection heads may be averaged to obtain a final positioning result of the target.

[0076] On the basis of the above embodiment, before extracting the feature map from the traffic scene image to be detected by the backbone network of the target detection model, the present embodiment further includes:

[0077] Inputting a traffic scene image sample into the backbone network of the target detection model, and extracting a feature map from the traffic scene image sample;

[0078] fusing the feature graph of the traffic scene image sample through a first fusion branch and a second fusion branch in a neck network of the target detection model;

[0079] Inputting the final fusion result of the feature maps of different depths of the traffic scene image sample by the feature fusion method into the corresponding detection head in the head network, and obtaining the classification result of the target in the traffic scene image sample output by one network branch in each detection head and the positioning result of the target in the traffic scene image sample output by another network branch;

[0080] Calculating the classification loss, the confidence loss of the positioning result, and the bounding box regression loss of the classification result of the target in the traffic scene image sample outputted by the two network branches in each detection head;

[0081] Determine the loss function of the target detection model according to the classification loss of the classification result of the target in the traffic scene image sample outputted by the two network branches in each detection head, the confidence loss of the positioning result and the bounding box regression loss;

[0082] The target detection model is trained according to the loss function.

[0083] The loss function of the target detection model includes the confidence loss L obj , classification loss L cls and the bounding box regression loss L CIOU Among them, the classification loss L cls The bounding box regression loss L is calculated based on the error between the model classification result and the calibration classification result of the target in the traffic scene image sample. CIOU The confidence loss L is calculated based on the error between the predicted box and the calibrated box of the target in the traffic scene image sample. obj Calculate the confidence of the model. The formula of the total loss function of the target detection model is as follows:

[0084]

[0085] Where K is the number of samples, , and is the weight parameter. Confidence loss L obj and classification loss L cls The calculation formula is as follows:

[0086]

[0087]

[0088] On the basis of the above embodiment, in this embodiment, the bounding box regression loss of the positioning result of the target in the traffic scene image sample is calculated, including:

[0089] The CIoU loss function is used to calculate the bounding box regression loss of the positioning result of the target.

[0090] The bounding box loss function has attracted extensive research attention because it affects the positioning accuracy of the detection results. The traditional positioning loss is to regress the coordinates of the four points independently, which has certain defects. The IoU loss regresses the coordinates of the four points of the bounding box as a whole, thus solving the above limitations. At present, the most widely used method is the CIoU loss function. The calculation process is shown in the formula. It has made certain improvements on the basis of the DIoU loss and improved the regression accuracy.

[0091]

[0092]

[0093]

[0094]

[0095] in, is the Euclidean distance between the predicted box and the true box, c is the diagonal length of the minimum enclosing box containing the predicted box and the true box, is a balance parameter used to balance the IoU loss and the aspect ratio loss. It is a measure of the difference in aspect ratio between the predicted box and the true box.

[0096] A large number of experiments were conducted on two traffic scene datasets and compared with other mainstream methods and advanced methods, showing the superior performance of the method proposed in this embodiment compared with other models. The codes of other comparison models in the experiments used widely recognized reproduction versions.

[0097] Table 1 shows the experimental results of different models on the TT100K dataset. The data in the table clearly shows the comparison. First, compared with the YOLOv5s baseline model, the invented model has a significant improvement in performance, with mAP increased by 21.4% and an accuracy of 84.2%, which shows that it effectively solves the limitations of the YOLOv5s baseline model in small object detection. In addition, the performance of this model far exceeds that of the two-stage model. Detection models with large parameters such as YOLOv5m and YOLOv7 have not shown good results in traffic scene data.

[0098] In addition, compared with some state-of-the-art object detection algorithms such as YOLOv7-Tiny and YOLOv8s, the proposed algorithm also outperforms them and achieves the best results. It is worth noting that the improved model only adds 4.8M parameters and still maintains the ability of real-time detection. Among lightweight models with similar parameters, this method has obvious advantages in small traffic object detection.

[0099] Table 1

[0100]

[0101] In order to further prove the effectiveness of the invented model, twelve mainstream target detection algorithms were selected and experiments were conducted on the BDD100K dataset. The results are shown in Table 2. Since the BDD100K dataset contains a large number of multi-scale targets, the focus of the experiment is to compare and evaluate small-scale targets. Therefore, the detection results (AP) of different algorithms for two types of small targets (traffic signs and traffic lights) in the dataset are specially listed. According to the data in the table, the AP values ​​of the invented detection algorithm for traffic signs and traffic lights reached 71.2% and 67.6% respectively, which is a significant performance improvement compared with other detection algorithms, and has achieved the best effect in small target detection. At the same time, the proposed algorithm still maintains a lightweight model parameter amount and has a speed that can perform real-time detection.

[0102] Table 2

[0103]

[0104] The above-mentioned large amount of experimental data shows that the model of the present invention is very suitable for detecting small targets in the traffic field. In addition, the performance advantages demonstrated on two different data sets further confirm its versatility in solving the problem of small target detection.

[0105] By using Grad-CAM, a visualization heat map of the target detection model and the YOLOv5s model in the present invention is generated, as shown in Figure 4 As shown in the figure. It can be observed that the YOLOv5s model is greatly affected by environmental noise. For the input image in the first row, the baseline model did not find the target, while the proposed model can still recognize the target. In the input image in the second row, the proposed model focuses on the target more accurately. It can be clearly seen from the visualized heat map that the method of the present invention is more sensitive to small targets and can accurately focus on the predetermined target.

[0106] The small target detection device in a traffic scene provided by the present invention is described below. The small target detection device in a traffic scene described below and the small target detection method in a traffic scene described above can be referenced to each other.

[0107] like Figure 5 As shown, the device includes an extraction module 501, a fusion module 502 and a detection module 503, wherein:

[0108] The extraction module 501 is used to extract a feature map from the traffic scene image to be detected through the backbone network of the target detection model;

[0109] The fusion module 502 is used to fuse the feature maps through the first fusion branch from deep to shallow layers and the second fusion branch from shallow to deep layers in the neck network of the target detection model, select the deepest two layers of feature maps of the backbone network and the shallowest feature map to be fused through the first fusion branch; select the sub-shallow feature map of the backbone network and the fusion result of the first fusion branch to be fused through the second fusion branch, and then fuse with the deepest feature map of the backbone network;

[0110] The detection module 503 is used to input the final fusion result of the feature fusion method used by the target detection model into the head network of the target detection model, and the head network detects the target from the traffic scene image to be detected.

[0111] This embodiment proposes a small target detection method for traffic scenes based on an efficient feature fusion network. On the one hand, layers are selected from the two fusion branches in the neck network for feature map fusion, thereby reducing information redundancy; on the other hand, the second fusion branch in the neck network can fuse the feature maps generated by the backbone network across the first fusion branch, strengthen the interaction of information flows between different levels, and significantly improve feature expression, thereby achieving more effective feature fusion and improving the detection accuracy of small targets in traffic scenes.

[0112] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor (processor) 610, a communication interface (Communications Interface) 620, a memory (memory) 630 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call the logic instructions in the memory 630 to execute the small target detection method in the traffic scene, which includes: extracting a feature map from the traffic scene image to be detected through the backbone network of the target detection model; fusing the feature map through the first fusion branch from deep to shallow and the second fusion branch from shallow to deep in the neck network of the target detection model, selecting the deepest two layers of feature maps of the backbone network and the shallowest feature map to be fused through the first fusion branch; on the basis of the first fusion branch, selecting the secondary shallow feature map of the backbone network and the final fusion result of the first fusion branch to be fused through the second fusion branch, and then fusing with the deepest feature map of the backbone network; inputting the final fusion result of the feature fusion method of the target detection model into the head network of the target detection model, and the head network detects the target from the traffic scene image to be detected.

[0113] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.

[0114] On the other hand, the present invention also provides a computer program product, which includes a computer program, which can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the small target detection method in the traffic scene provided by the above methods, which method includes: extracting a feature map from the traffic scene image to be detected through the backbone network of the target detection model; fusing the feature map through the first fusion branch from deep to shallow and the second fusion branch from shallow to deep in the neck network of the target detection model, selecting the deepest two layers of feature maps of the backbone network and the shallowest feature map to be fused through the first fusion branch; on the basis of the first fusion branch, selecting the secondary shallow feature map of the backbone network and the final fusion result of the first fusion branch to be fused through the second fusion branch, and then fusing with the deepest feature map of the backbone network; inputting the final fusion result of the feature fusion method of the target detection model into the head network of the target detection model, and the head network detects the target from the traffic scene image to be detected.

[0115] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to execute the small target detection method in the traffic scene provided by the above-mentioned methods, the method comprising: extracting a feature map from the traffic scene image to be detected through the backbone network of the target detection model; fusing the feature map through the first fusion branch from deep to shallow and the second fusion branch from shallow to deep in the neck network of the target detection model, selecting the deepest two layers of feature maps of the backbone network and the shallowest feature map to be fused through the first fusion branch; on the basis of the first fusion branch, selecting the secondary shallow feature map of the backbone network and the final fusion result of the first fusion branch to be fused through the second fusion branch, and then fusing with the deepest feature map of the backbone network; inputting the final fusion result of the feature fusion method of the target detection model into the head network of the target detection model, and the head network detects the target from the traffic scene image to be detected.

[0116] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0117] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting small targets in traffic scenes, characterized in that: include: Extract feature maps from the traffic scene image to be detected through the backbone network of the target detection model; The feature maps are fused through a first fusion branch from deep to shallow layers and a second fusion branch from shallow to deep layers in the neck network of the target detection model, and the deepest two layers of feature maps of the backbone network and the shallowest layer of feature maps are selected and fused through the first fusion branch; Select the fusion result of the second shallow feature map of the backbone network and the first fusion branch, fuse them through the second fusion branch, and then fuse them with the deepest feature map of the backbone network; The final fusion result of the feature fusion method used by the target detection model is input into the head network of the target detection model, and the head network detects the target from the traffic scene image to be detected.

2. The method for detecting small targets in traffic scenes according to claim 1, characterized in that: Before fusing the feature map through a first fusion branch from a deep layer to a shallow layer and a second fusion branch from a shallow layer to a deep layer in the neck network of the target detection model, the method further includes: The similarity between the fusion result of each current layer in the neck network and the fusion result of the previous layer of the current layer is calculated respectively, the feature layer in the first fusion branch of the neck network is connected one-to-one with the feature layer in the trunk network of the same depth, and the feature layer in the second fusion branch is connected one-to-one with the feature layer in the first fusion branch of the same depth; The current layer whose similarity is greater than a preset threshold is deleted.

3. The method for detecting small targets in traffic scenes according to claim 1, characterized in that: The feature maps are fused through a first fusion branch from a deep layer to a shallow layer in a neck network of the target detection model, including: The first fusion branch uses a bilinear interpolation method to upsample the feature map and then fuse it.

4. The method for detecting small targets in traffic scenes according to claim 1, characterized in that: Inputting the final fusion result of the feature fusion method used by the target detection model into the head network of the target detection model, and detecting the target from the traffic scene image to be detected by the head network, including: The fusion results of different depths of the feature fusion method used by the target detection model are respectively input into the corresponding detection heads in the head network, and a convolution operation is performed through two network branches in the detection head to obtain the classification result of the target output by one network branch of each detection head and the positioning result of the target output by another network branch; Obtaining a final classification result of the target according to the classification result of the target output by the detection head; The final positioning result of the target is obtained according to the positioning result of the target output by the detection head.

5. The method for detecting small targets in traffic scenes according to claim 4, characterized in that: Before extracting the feature map from the traffic scene image to be detected through the backbone network of the target detection model, it also includes: Inputting a traffic scene image sample into the backbone network of the target detection model, and extracting a feature map from the traffic scene image sample; fusing the feature graph of the traffic scene image sample through a first fusion branch and a second fusion branch in a neck network of the target detection model; Inputting the final fusion result of the feature maps of different depths of the traffic scene image sample by the feature fusion method into the corresponding detection head in the head network, and obtaining the classification result of the target in the traffic scene image sample output by one network branch in each detection head and the positioning result of the target in the traffic scene image sample output by another network branch; Calculating the classification loss, the confidence loss of the positioning result, and the bounding box regression loss of the classification result of the target in the traffic scene image sample outputted by the two network branches in each detection head; Determine the loss function of the target detection model according to the classification loss of the classification result of the target in the traffic scene image sample outputted by the two network branches in each detection head, the confidence loss of the positioning result and the bounding box regression loss; The target detection model is trained according to the loss function.

6. The method for detecting small targets in traffic scenes according to claim 5, characterized in that: Calculating the bounding box regression loss of the positioning result of the target in the traffic scene image sample includes: The CIoU loss function is used to calculate the bounding box regression loss of the positioning result of the target.

7. A small target detection device in a traffic scene, characterized in that: include: An extraction module, used for extracting a feature map from a traffic scene image to be detected through a backbone network of a target detection model; A fusion module, used to fuse the feature maps through a first fusion branch from deep to shallow layers and a second fusion branch from shallow to deep layers in the neck network of the target detection model, and select the deepest two layers of feature maps of the backbone network and the shallowest layer of feature maps to be fused through the first fusion branch; Select the second shallow feature map of the backbone network and the final fusion result of the first fusion branch to be fused through the second fusion branch, and then fuse with the deepest feature map of the backbone network; The detection module is used to input the final fusion result of the feature fusion method used by the target detection model into the head network of the target detection model, and the head network detects the target from the traffic scene image to be detected.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for detecting small targets in traffic scenes as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for detecting small targets in traffic scenes as claimed in any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for detecting small targets in traffic scenes as claimed in any one of claims 1 to 6 is implemented.

Images