Target detection method and device in complex traffic scene and computer device

Abstract

The application relates to a target detection method, device and computer equipment under a complex traffic scene. The method comprises the following steps: determining a detection result of a to-be-detected image based on a traffic target detection model; a mixed encoder layer of the traffic target detection model comprises an FDPN-P2 network; the detection result of the to-be-detected image is determined based on the traffic target detection model, which comprises the following steps: inputting original features obtained by processing the to-be-detected image into a plurality of residual blocks of a backbone network to obtain scale features,, and ; based on a feature fusion unit and a RepC3 unit of the FDPN-P2 network, the scale features and and the scale features after self-attention processing are subjected to fusion processing to obtain multi-scale features, and, and the multi-scale features after preprocessing and the scale features are fused to obtain multi-scale features; a plurality of Tokens are obtained based on the multi-scale features,, and, and a detection result of the to-be-detected image is obtained based on the plurality of Tokens. The method can improve the reliability of target detection under a complex traffic scene.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular to a target detection method, device, and computer equipment in complex traffic scenarios. Background Technology

[0002] With the rapid development of intelligent transportation systems, environmental perception technology based on sensing units has become an important foundation for supporting autonomous driving, vehicle-road cooperation, and traffic operation monitoring and management. Among them, target detection technology uses sensing devices such as cameras and radars deployed along roads to detect and identify traffic participants such as vehicles, pedestrians, and non-motorized vehicles in real time, providing key data support for autonomous driving, traffic signal control, accident early warning, violation detection, and collaborative decision-making.

[0003] Existing traffic target detection methods are mostly based on the RT-DETR (Real-Time Detection Transformer) model. However, the query selection and regression mechanism unique to the RT-DETR model is not friendly to the detection of small targets, making it difficult to identify small targets and objects under limited visual conditions in complex road environments. This easily leads to the problem of missed detection of traffic targets, thereby reducing the reliability of complex traffic target detection. Summary of the Invention

[0004] Therefore, it is necessary to address the aforementioned technical problems by providing a method, device, and computer equipment for target detection in complex traffic scenarios that can accurately identify small targets and objects under limited visual conditions in complex road environments, avoid missing traffic targets, and thus improve the reliability of target detection in complex traffic scenarios.

[0005] Firstly, this application provides a target detection method for complex traffic scenarios, including:

[0006] The traffic target detection model is based on a pre-trained traffic target detection model to determine the detection results of traffic targets in the image to be tested. The traffic target detection model includes a backbone network, a hybrid encoder layer, a perceptual query selection layer, and a detection head layer. The hybrid encoder layer includes the FDPN-P2 network.

[0007] The process of determining the detection results of traffic targets in the image to be tested based on the traffic target detection model includes:

[0008] The backbone network is used to perform convolution and pooling on the image under test to obtain the original features. These original features are then input into multiple residual blocks of the backbone network to obtain scale features. , , and ;

[0009] Based on multiple feature fusion units and multiple RepC3 units in the FDPN-P2 network, scale features are optimized. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and ;

[0010] In the FDPN-P2 network, the upsampled multi-scale features and scale features Feature concatenation and feature fusion are performed to obtain multi-scale features. ;

[0011] Based on the perceptual query selection layer for multi-scale features , , and Multiple tokens are obtained by performing feature flattening, and these tokens are then processed based on the perception query selection layer and the detection head layer to obtain the detection results of traffic targets.

[0012] In one embodiment, the residual block includes a main path and shortcut branches; the original features are input into multiple residual blocks of the backbone network to obtain scale features. , , and ,include:

[0013] For each residual block, in the main path of that residual block, the original features are sequentially processed based on the first feature convolutional layer and the second feature convolutional layer to obtain the input features. And based on the main path BAM, the input features are analyzed. Feature enhancement processing is performed to obtain refined features. ;

[0014] Based on the shortcut branch of the residual block, determine the step size and channel dimension of the residual block and the refinement features. Whether the step size and channel dimension match;

[0015] When the step size and channel dimension of the residual block are determined, along with the refinement features... When the step size and channel dimension do not match, in the shortcut branch, average pooling and channel alignment are performed on the original features to obtain aligned original features; then the aligned original features are combined with the refined features. Element-by-element addition is performed to obtain a scale feature;

[0016] When the step size and channel dimension of the residual block are determined, along with the refinement features... When matching the step size and channel dimension, the original features are compared with the refined features. By performing element-wise addition, a scale feature is obtained.

[0017] In one embodiment, the main path-based BAM is used to analyze the input features. Feature enhancement processing is performed to obtain refined features. ,include:

[0018] For input features The input features are obtained by performing average pooling, multilayer sensing, and normalization. Corresponding channel attention features Channel attention features The calculation formula is ,in, Indicates batch normalization. This represents a multilayer perceptron. Indicates average pooling. and These represent the weights of the dimensionality reduction layer and the weights of the dimensionality increase layer, respectively. and These represent the bias parameters of the dimensionality reduction layer and the bias parameters of the dimensionality increase layer, respectively.

[0019] For input features Convolution and normalization are performed to obtain the input features. Corresponding spatial attention features Spatial attention characteristics The calculation formula is ,in, This represents a 1×1 convolutional kernel located in the first convolutional layer. This represents a 3×3 convolutional kernel located in the second convolutional layer. This represents a 3×3 convolutional kernel located in the third convolutional layer. This represents a 1×1 convolutional kernel located in the fourth convolutional layer;

[0020] Channel attention features Spatial attention features Weighted fusion processing is performed to obtain a 3D attention map. 3D attention map The calculation formula is ,in, It is the sigmoid function;

[0021] Based on 3D attention map For input features After processing, the refined features are obtained. .

[0022] In one embodiment, the FDPN-P2 network includes a self-attention processing unit; based on multiple feature fusion units and multiple RepC3 units in the FDPN-P2 network, scale features are processed. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and ,include:

[0023] Based on self-attention processing unit for scale features Self-attention processing is performed to obtain attention-enhancing features. ;

[0024] Based on the first feature fusion unit, scale features , and attention enhancement features Feature processing is performed to obtain mesoscale features. ;

[0025] Mesoscale features based on FDPN-P2 network The first upsampled feature is obtained by performing upsampling processing. The first upsampled feature Scale features The first stitched feature is obtained by performing a stitching process; and the mesoscale feature is then processed. The first convolutional feature is obtained by performing 3×3 convolution. , the first convolutional features Scale features The splicing process is performed to obtain the second splicing feature;

[0026] The first splicing feature is obtained by fusing and refining the first splicing feature using the first RepC3 unit, and the second splicing feature is obtained by fusing and refining the second splicing feature using the second RepC3 unit.

[0027] Based on the second feature fusion unit, mesoscale features The first and second fusion features are fused to obtain multi-scale features. .

[0028] In one embodiment, the method further includes:

[0029] Multi-scale features based on FDPN-P2 network The second upsampled feature is obtained by performing upsampling processing. ;

[0030] Second upsampling features based on FDPN-P2 network First upsampling feature The first fusion feature is then spliced ​​together to obtain the third spliced ​​feature;

[0031] The third RepC3 unit is used to fuse and refine the third spliced ​​features to obtain multi-scale features. .

[0032] In one embodiment, the method further includes:

[0033] Multi-scale features based on FDPN-P2 network Perform 3×3 convolution to obtain the second convolution feature. ;

[0034] Based on the FDPN-P2 network, the second convolutional features are analyzed. First convolutional features The second fusion feature is then spliced ​​together to obtain the fourth splicing feature;

[0035] The fourth RepC3 unit is used to fuse and refine the fourth spliced ​​feature to obtain multi-scale features. .

[0036] In one embodiment, scale features are fused based on the first feature fusion unit in the FDPN-P2 network. , and attention enhancement features Feature processing is performed to obtain mesoscale features. ,include:

[0037] A combination of downsampling, 1×1 convolution, and upsampling operations are used to process scale features. , and attention enhancement features Spatial resolution and channel number alignment are performed to obtain aligned features, and the aligned features are then stitched together along the channel dimension to obtain aligned and stitched features.

[0038] Parallel depthwise convolution processing is performed on the aligned and concatenated features using multiple sets of convolution kernels of different sizes. The outputs of each parallel depthwise convolution processing branch are then weighted and superimposed with the aligned and concatenated features to obtain the superimposed features.

[0039] The superimposed features are processed by pointwise convolution, and the mesoscale features are output through residual connections. .

[0040] In one embodiment, the pre-training process of the traffic target detection model includes:

[0041] Traffic targets in training images are detected based on a pre-acquired initial target detection model to obtain training prediction boxes;

[0042] Based on the training predicted bounding boxes and the corresponding ground truth bounding boxes in the training images, determine the area of ​​the overlapping rectangle between the training predicted bounding boxes and the ground truth bounding boxes. and the area of ​​union ; where the area of ​​the overlapping rectangles The calculation formula is , , , and These represent the right, left, bottom, and top boundaries of the true bounding box, respectively. , , and These represent the right, left, bottom, and top boundaries of the training prediction box, respectively. This represents the smallest numerical value between the boundaries of the training predicted bounding box and the boundaries of the ground truth bounding box. The area of ​​the union of the training predicted bounding box and the ground truth bounding box represents the largest numerical value; The calculation formula is , and These represent the width and height of the training prediction box, respectively. and These represent the width and height of the actual bounding box, respectively. Scale factor;

[0043] Based on the area of ​​overlapping rectangles Area of ​​union The intersection-union ratio (IU) between the scaled-down training prediction boxes and the ground truth boxes is determined to be . ;

[0044] Based on the cross-union ratio of the shrunk boxes, the SIoU loss value of the unshrunk training predicted boxes, and the cross-union ratio between the unshrunk training predicted boxes and the ground truth boxes, the objective loss function is determined as follows: , The SIoU loss value is the training prediction box without scaling. The intersection-union ratio (IU) between the unscaled training predicted boxes and the ground truth boxes;

[0045] The initial target detection model is optimized based on the objective loss function to obtain the traffic target detection model.

[0046] Secondly, this application also provides an XXX device, comprising:

[0047] The target detection module is used to determine the detection results of traffic targets in the image to be tested based on a pre-trained traffic target detection model. The traffic target detection model includes a backbone network, a hybrid encoder layer, a perceptual query selection layer, and a detection head layer. The hybrid encoder layer includes the FDPN-P2 network.

[0048] The process of determining the detection results of traffic targets in the image to be tested based on the traffic target detection model includes:

[0049] The backbone network is used to perform convolution and pooling on the image under test to obtain the original features. These original features are then input into multiple residual blocks of the backbone network to obtain scale features. , , and ;

[0050] Based on multiple feature fusion units and multiple RepC3 units in the FDPN-P2 network, scale features are optimized. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and ;

[0051] In the FDPN-P2 network, the upsampled multi-scale features and scale features Feature concatenation and feature fusion are performed to obtain multi-scale features. ;

[0052] Based on the perceptual query selection layer for multi-scale features , , and Multiple tokens are obtained by performing feature flattening, and these tokens are then processed based on the perception query selection layer and the detection head layer to obtain the detection results of traffic targets.

[0053] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0054] The traffic target detection model is based on a pre-trained traffic target detection model to determine the detection results of traffic targets in the image to be tested. The traffic target detection model includes a backbone network, a hybrid encoder layer, a perceptual query selection layer, and a detection head layer. The hybrid encoder layer includes the FDPN-P2 network.

[0055] The process of determining the detection results of traffic targets in the image to be tested based on the traffic target detection model includes:

[0056] The backbone network is used to perform convolution and pooling on the image under test to obtain the original features. These original features are then input into multiple residual blocks of the backbone network to obtain scale features. , , and ;

[0057] Based on multiple feature fusion units and multiple RepC3 units in the FDPN-P2 network, scale features are optimized. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and ;

[0058] In the FDPN-P2 network, the upsampled multi-scale features and scale features Feature concatenation and feature fusion are performed to obtain multi-scale features. ;

[0059] Based on the perceptual query selection layer for multi-scale features , , and Multiple tokens are obtained by performing feature flattening, and these tokens are then processed based on the perception query selection layer and the detection head layer to obtain the detection results of traffic targets.

[0060] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0061] The traffic target detection model is based on a pre-trained traffic target detection model to determine the detection results of traffic targets in the image to be tested. The traffic target detection model includes a backbone network, a hybrid encoder layer, a perceptual query selection layer, and a detection head layer. The hybrid encoder layer includes the FDPN-P2 network.

[0062] The process of determining the detection results of traffic targets in the image to be tested based on the traffic target detection model includes:

[0063] The backbone network is used to perform convolution and pooling on the image under test to obtain the original features. These original features are then input into multiple residual blocks of the backbone network to obtain scale features. , , and ;

[0064] Based on multiple feature fusion units and multiple RepC3 units in the FDPN-P2 network, scale features are optimized. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and ;

[0065] In the FDPN-P2 network, the upsampled multi-scale features and scale features Feature concatenation and feature fusion are performed to obtain multi-scale features. ;

[0066] Based on the perceptual query selection layer for multi-scale features , , and Multiple tokens are obtained by performing feature flattening, and these tokens are then processed based on the perception query selection layer and the detection head layer to obtain the detection results of traffic targets.

[0067] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0068] The traffic target detection model is based on a pre-trained traffic target detection model to determine the detection results of traffic targets in the image to be tested. The traffic target detection model includes a backbone network, a hybrid encoder layer, a perceptual query selection layer, and a detection head layer. The hybrid encoder layer includes the FDPN-P2 network.

[0069] The process of determining the detection results of traffic targets in the image to be tested based on the traffic target detection model includes:

[0070] The backbone network is used to perform convolution and pooling on the image under test to obtain the original features. These original features are then input into multiple residual blocks of the backbone network to obtain scale features. , , and ;

[0071] Based on multiple feature fusion units and multiple RepC3 units in the FDPN-P2 network, scale features are optimized. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and ;

[0072] In the FDPN-P2 network, the upsampled multi-scale features and scale features Feature concatenation and feature fusion are performed to obtain multi-scale features. ;

[0073] Based on the perceptual query selection layer for multi-scale features , , and Multiple tokens are obtained by performing feature flattening, and these tokens are then processed based on the perception query selection layer and the detection head layer to obtain the detection results of traffic targets.

[0074] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: In the present invention, the FDPN-P2 network uses multiple feature fusion units and multiple RepC3 units to process scale features. and and scale features after self-attention processing By performing fusion processing, each scale feature can possess detailed contextual information, and features at different scales can be fused together, mitigating the problem of semantic loss; multi-scale features It retains the most accurate pixel-level edges and details, boasting the highest resolution, rich detail and location information, and excellent features for detecting small targets; therefore, based on multi-scale features... and multi-scale features , and Multiple tokens are obtained and processed based on the perceptual query selection layer and the detection head layer to obtain the detection results of traffic targets. This can explicitly expand the scale range of the feature pyramid and realize the comprehensive utilization of low-level detailed information and high-level semantic information, thereby significantly improving the traffic target detection model's ability to recognize small-sized targets and objects under limited visual conditions. This effectively avoids the phenomenon of missing targets in complex traffic scenes and improves the reliability of target detection. Attached Figure Description

[0075] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0076] Figure 1This is a schematic diagram of the process for determining traffic target detection results based on a traffic target detection model in one embodiment;

[0077] Figure 2 This is a flowchart illustrating a traffic target detection model in one embodiment;

[0078] Figure 3 This is a flowchart illustrating the BAM process in one embodiment.

[0079] Figure 4 This is a flowchart illustrating the feature fusion unit in one embodiment;

[0080] Figure 5 This is a schematic diagram comparing the performance indicators of a traffic target detection model and a traditional RT-DETR model in one embodiment.

[0081] Figure 6 This is a schematic diagram of the metrics for each target category in an RT-DETR model in one embodiment;

[0082] Figure 7 This is a schematic diagram of the indicators for each target category in a traffic target detection model in one embodiment;

[0083] Figure 8 This is a structural block diagram of a target detection device in a complex traffic scenario in one embodiment. Detailed Implementation

[0084] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0085] The RT-DETR (Real-Time Detection Transformer) model is an end-to-end object detection model based on Transformer. However, its detection accuracy is often lower than that of highly optimized YOLO (You Only Look Once) series object detection algorithms when dealing with extremely small objects. Specifically, in complex road scenes, the RT-DETR model's internal CCFM (Cross-Scale Feature Fusion Module) still exhibits insufficient perception accuracy when facing small-sized objects, especially in high-density traffic flow. Occluded targets have extremely limited visible areas, resulting in a severe lack of effective distinguishing features. Furthermore, small targets at a distance have a very low pixel count, leading to severe feature dilution in the top-level information flow of the multi-scale feature pyramid. In addition, the RT-DETR model relies on classification scores to select K features to initialize the query vector. Small targets, due to feature ambiguity, typically have low confidence and are easily squeezed out of the candidate list by large targets or significant background in the first step, leading to missed detections. Meanwhile, the Transformer's direct coordinate regression method is extremely sensitive to positional bias. For tiny objects, a prediction error of just a few pixels can cause a sharp drop in IoU (Intersection over Union), making gradients unstable during training and hindering pixel-level precision in localization compared to anchor-based object detection algorithms. These factors collectively lead to the RT-DETR model missing key traffic participants and falsely detecting similar background textures during inference, making it difficult to meet the safety redundancy requirements of autonomous driving or intelligent perception systems across all scenarios and weather conditions. Therefore, this application focuses on extracting features of small-sized and occluded targets, aiming to improve the detection accuracy of road target detection models in complex road scenes. Specifically, addressing the problem of missed or false detections of vehicles and pedestrians in real-world complex road environments, this application integrates the RTDETR-FP2 traffic target detection model (i.e., the FDPN (focusing diffusion pyramid network)) and the P2 detection head, improving the model's ability to detect small targets. Furthermore, this traffic target detection model introduces a BAM (Bottleneck Attention Module) into the backbone network. The resulting BAM residual module effectively focuses on relevant information in the image, avoiding interference from irrelevant information. Finally, the traffic target detection model introduces the Inner-SIoU loss function, using a scale factor to control the size of the auxiliary bounding box to optimize loss calculation, thus improving the detection capability for fine targets.

[0086] In an exemplary embodiment, a target detection method for complex traffic scenes is provided. The method includes: determining the detection result of traffic targets in a test image based on a pre-trained traffic target detection model; the traffic target detection model includes a backbone network, a hybrid encoder layer, a perceptual query selection layer, and a detection head layer, wherein the hybrid encoder layer includes an FDPN-P2 network; wherein, as... Figure 1 The diagram shows a flowchart for determining traffic target detection results based on a traffic target detection model. The flowchart includes steps 102 to 108.

[0087] Step 102: Based on the backbone network, perform convolution and pooling processing on the image to be tested to obtain the original features. Input the original features into multiple residual blocks of the backbone network to obtain scale features. , , and .

[0088] For example, in traffic target detection in complex traffic scenarios, the image to be tested can come from video streams from roadside or vehicle-mounted cameras, which can be obtained by frame extraction to obtain single-frame images. Furthermore, the image to be tested is obtained by preprocessing the aforementioned single-frame images. The preprocessing process can include decoding, proportional scaling, pixel normalization, format conversion, and padding to a fixed size (e.g., 640*640). Preprocessing the single-frame images yields a uniformly sized and stable image to be tested. Using this image as input to the traffic target detection model ensures stable and efficient detection.

[0089] Scale features , , and These represent the feature maps output by the backbone network at different stages. The features at each scale can be understood as single-scale features; specifically, scale features... This can be understood as shallow features, which have high spatial resolution and retain rich details such as edges and textures, making them suitable for describing small-scale targets such as pedestrians and distant vehicles; scale features and These can be understood as mid-level features, striking a balance between spatial resolution and semantic expressive power, effectively representing medium-sized targets; scale features. This can be understood as deep features, with the lowest spatial resolution but the most abstract semantic information. It can highlight the overall structure and category attributes of the target, making it suitable for describing large-scale targets such as large vehicles and road areas. Therefore, scale features... , , and The corresponding features at different scales are feature representations of the same input image formed at different network layers, with different emphases on target size and semantic information. The collaborative use of features at multiple scales can effectively improve the accuracy and robustness of traffic target detection in complex traffic scenarios.

[0090] Optionally, frames are extracted from the video stream of a roadside or vehicle-mounted camera to obtain single-frame images. These single-frame images are then preprocessed to obtain the image to be tested. This image is then input into a traffic target detection model. The backbone network of the traffic target detection model first performs convolution and pooling processing on the image to obtain raw features. These raw features are then input into multiple residual blocks to obtain scale features. , , and Each residual block includes a main path and a shortcut branch. The main path and the shortcut branch process the original features and form a residual structure. Then, the output of the main path and the output of the shortcut branch are added element by element to fuse deep semantics and shallow details. After activation, a scale feature is output.

[0091] Step 104: Based on multiple feature fusion units and multiple RepC3 units in the FDPN-P2 network, scale features are processed. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and .

[0092] The FDPN-P2 network proposed in this application is a type of network with FDPN (Focused Diffusion Pyramid Network) as the main structure, which features the scale characteristics of the backbone network output. , and Multi-round cross-scale fusion is performed to enhance semantic interaction between features at different scales; and high-resolution multi-scale features are introduced on this basis. This involves constructing multi-scale feature representations that incorporate fine-grained information and high-level semantic information, thereby improving the detection performance of small and distant targets in traffic scenarios. Specifically, in the FDPN-P2 network, a feature fusion unit is introduced, and... Layered features enhance the interaction between features at different scales, effectively improving the detection capability of small targets and explicitly expanding the scale range of the feature pyramid, thereby improving the accuracy of traffic target detection in complex traffic scenarios. The feature fusion unit focuses on scale features extracted from the backbone network. , and This is used to give features detailed contextual information at each scale and to fuse features across different scales.

[0093] The RepC3 (Re-parameterized C3) unit is used to further integrate, compress, and enhance the representation of fused features while maintaining inference efficiency. Multi-scale features. , and All features are integrated from different scales. Through feature fusion units and RepC3 units, the features at each scale retain shallow details and enhance deep semantics, thus forming a set of highly robust multi-scale features that can effectively support roadside target detection in small targets and complex environments.

[0094] For example, please refer to Figure 2 , Figure 2 This is a flowchart illustrating a traffic target detection model. It shows the scale features output by the backbone network. , , and As input to the hybrid encoder layer, where only scale features are considered. Self-attention processing is performed to obtain attention-enhancing features. Attention-enhancing features based on the first feature fusion unit and scale features and Processing is performed to generate mesoscale features. And based on mesoscale features Downsampling and upsampling operations are performed separately. Downsampling extracts features through 3×3 convolution and then fuses them with higher-level features via channel concatenation (concat). Upsampling enlarges the feature map size and then fuses it with lower-level features via channel concatenation. Subsequently, the fused features are initially integrated using a RepC3 unit to generate scale features. and attention enhancement features The corresponding intermediate feature branches; and the scale features and attention enhancement features The corresponding intermediate feature branches and mesoscale features, respectively The second feature fusion unit is input to perform secondary deep feature fusion, further enhancing the multi-scale semantic representation and obtaining multi-scale features. Furthermore, by utilizing three-way cross-layer skip connections, the fused features are integrated again through the RepC3 unit to output the final highly robust multi-scale features. and .

[0095] Step 106: In the FDPN-P2 network, process the upsampled multi-scale features. and scale features Feature concatenation and feature fusion are performed to obtain multi-scale features. .

[0096] Optionally, in the FDPN-P2 network, multi-scale features are... Perform another upsampling process, and then use the concat function to concatenate the upsampled multi-scale features. and scale features Feature splicing is performed, and finally the fifth RepC3 unit (i.e. Figure 2 (5RepC3) will be used to perform the multi-scale feature upsampling process after the above concatenation. and scale features By integrating the features, multi-scale features are obtained. .

[0097] Step 108, based on the perceptual query selection layer, select multi-scale features. , , and Multiple tokens are obtained by performing feature flattening, and these tokens are then processed based on the perception query selection layer and the detection head layer to obtain the detection results of traffic targets.

[0098] The detection results of traffic targets can include the category of each traffic target, the boundary of each traffic target in the image, and its corresponding confidence score. Traffic targets can include vehicles, pedestrians, non-motorized vehicles, etc.

[0099] The perceptual query selection layer selects multiple high-quality feature points based on the predicted IoU scores of multiple feature points (Tokens). It then acquires the spatial location and content features corresponding to these high-quality feature points and uses them as initial object query vectors. These initial object query vectors are input into the decoder of the detection head layer. A self-attention mechanism is used to model the relationship between the initial object query vectors, and a cross-attention mechanism guides each initial object query vector to extract fine-grained features related to the target from the feature map output by the encoder. The positional and semantic information of each initial object query vector is updated layer by layer to obtain the updated object query vector. The updated object query vector is then input into the detection head layer, and parallel MLP branches output the category prediction results and bounding box prediction results corresponding to the traffic target.

[0100] Optionally, multi-scale features , , and The data is fed into a perceptual query selection layer, which processes multi-scale features. , , and The sequence is flattened into a one-dimensional sequence containing thousands of feature points (tokens). These tokens are then fed into a perceptual query selection layer, which directly evaluates them and selects the top-K feature points (high-quality feature points) based on the predicted IoU scores of multiple tokens. The spatial location and content features of these top-K feature points are directly used as the initial object queues. Next, these initialized object queues carrying prior information enter the decoder of the detection head layer. Through self-attention and cross-attention mechanisms, they interact to extract fine details such as edges and textures of traffic targets, iteratively refining the position and semantic information of the initialized object queues layer by layer to obtain updated object queues. Finally, the updated object queues are fed into the detection head layer, where parallel MLP branches linearly map the high-dimensional features into the final class probabilities and bounding box coordinates, directly yielding the traffic target detection results.

[0101] In the aforementioned target detection methods for complex traffic scenarios, the FDPN-P2 network utilizes multiple feature fusion units and multiple RepC3 units to process scale features. and and scale features after self-attention processing By performing fusion processing, each scale feature can possess detailed contextual information, and features at different scales can be fused together, mitigating the problem of semantic loss; multi-scale features It retains the most accurate pixel-level edges and details, boasting the highest resolution, rich detail and location information, and excellent features for detecting small targets; therefore, based on multi-scale features... and multi-scale features , and Multiple tokens are obtained and processed based on the perceptual query selection layer and the detection head layer to obtain the detection results of traffic targets. This can explicitly expand the scale range of the feature pyramid and realize the comprehensive utilization of low-level detailed information and high-level semantic information, thereby significantly improving the traffic target detection model's ability to recognize small-sized targets and objects under limited visual conditions. This effectively avoids the phenomenon of missing targets in complex traffic scenes and improves the reliability of target detection.

[0102] In an exemplary embodiment, the residual block includes a main path and shortcut branches; the original features are input into multiple residual blocks of the backbone network to obtain scale features. , , and This includes: for each residual block, in the main path of that residual block, sequentially processing the original features based on the first feature convolutional layer and the second feature convolutional layer to obtain the input features. And based on the main path BAM, the input features are analyzed. Feature enhancement processing is performed to obtain refined features. Based on the shortcut branch of the residual block, determine the step size and channel dimension of the residual block and the refinement features. Whether the step size and channel dimension match; when it is determined that the step size and channel dimension of the residual block match the refinement feature. When the step size and channel dimension do not match, in the shortcut branch, average pooling and channel alignment are performed on the original features to obtain aligned original features; then the aligned original features are combined with the refined features. Element-wise addition is performed to obtain a scale feature; when the step size and channel dimension of the residual block are determined, the refinement feature is... When matching the step size and channel dimension, the original features are compared with the refined features. By performing element-wise addition, a scale feature is obtained.

[0103] The first feature convolutional layer is branch2a (feature extraction and activation) convolutional layer, and the second feature convolutional layer is branch2b (pure feature extraction) convolutional layer.

[0104] Optionally, frames are extracted from the video stream of a roadside or vehicle-mounted camera to obtain single-frame images. These single-frame images are then preprocessed to obtain the image to be tested. This image is then input into a traffic target detection model. The backbone network of the traffic target detection model first performs convolution and pooling processing on the image to obtain raw features. These raw features are then input into multiple residual blocks to obtain scale features. , , and Specifically, each residual block includes a main path and shortcut branches. The process for processing the original features in one residual block includes the following steps: In the main path of the residual block, the original features are sequentially processed based on the branch2a (feature extraction and activation) convolutional layer and the branch2b (pure feature extraction) convolutional layer to obtain the input features. and the input feature The input features are fed into the BAM in the main path, where the focus is on the input features. Based on the useful information in the image under test, the output of the main path is obtained, which is the refinement of features. The shortcut branch is determined and switched based on the step size and channel dimension of the residual block. Specifically, it determines whether the step size and channel dimension of the residual block are related to the refinement features. If the step size and channel dimension match, the shortcut branch directly passes the original feature, yielding the output of the shortcut branch; otherwise, if the step size and channel dimension of the residual block match the refined feature, the output of the shortcut branch is obtained. If the stride and channel dimensions do not match, average pooling is used to compress the size of the original features, followed by a 1*1 convolution to make the original features and the refined features more similar. Complete channel alignment to obtain the output of the shortcut branch. Then, refine the main path's output (features). The output of the shortcut branch is added element-wise to fuse deep semantics and shallow details, and then a scale feature is output after activation function.

[0105] In the previous exemplary embodiment, the main path-based BAM was used to analyze the input features. Feature enhancement processing is performed to obtain refined features. This includes: input features The input features are obtained by performing average pooling, multilayer sensing, and normalization. Corresponding channel attention features Channel attention features The calculation formula is ,in, Indicates batch normalization. This represents a multilayer perceptron. Indicates average pooling. and These represent the weights of the dimensionality reduction layer and the weights of the dimensionality increase layer, respectively. and These represent the bias parameters of the dimensionality reduction layer and the bias parameters of the dimensionality increase layer, respectively; for the input features Convolution and normalization are performed to obtain the input features. Corresponding spatial attention features Spatial attention characteristics The calculation formula is ,in, This represents a 1×1 convolutional kernel located in the first convolutional layer. This represents a 3×3 convolutional kernel located in the second convolutional layer. This represents a 3×3 convolutional kernel located in the third convolutional layer. This represents a 1×1 convolutional kernel located in the fourth convolutional layer; channel attention features. Spatial attention features Weighted fusion processing is performed to obtain a 3D attention map. 3D attention map The calculation formula is ,in, It is the sigmoid function; based on a 3D attention map. For input features After processing, the refined features are obtained. .

[0106] like Figure 3 The diagram shows the flow chart of BAM. BAM generates a feature set that is consistent with the input features. For attention maps of the same size (values ​​between 0 and 1), calculate the channel attention features separately. Spatial attention features .

[0107] For input features BAM will infer a three-dimensional attention map. And based on the 3D attention map For input features After processing, the refined features are obtained. Among them, the three-dimensional attention map The result is obtained through the following calculation:

[0108]

[0109] in, It is the sigmoid function. For channel attention features, Spatial attention features. Channel attention features. The calculation formula is as follows:

[0110]

[0111] In the above formula, Indicates batch normalization. This represents a multilayer perceptron. Indicates average pooling. and These represent the weights of the dimensionality reduction layer and the weights of the dimensionality increase layer, respectively. and These represent the bias parameters of the dimensionality reduction layer and the bias parameters of the dimensionality increase layer, respectively.

[0112] Spatial attention characteristics The calculation formula is as follows:

[0113]

[0114] In the above formula, This represents a 1×1 convolutional kernel located in the first convolutional layer. This represents a 3×3 convolutional kernel located in the second convolutional layer. This represents a 3×3 convolutional kernel located in the third convolutional layer. This represents a 1×1 convolutional kernel located in the fourth convolutional layer.

[0115] In this embodiment, the BAM attention mechanism in the backbone network can be used to form BAM residual blocks, which can then be used to effectively focus on the effective information in the image to be tested. That is, the traffic target detection model can "selectively" focus on the more important information in the features corresponding to the image to be tested, while suppressing unimportant information and avoiding interference from irrelevant information. This significantly improves the performance of the traffic target detection model while reducing computational costs.

[0116] In one exemplary embodiment, the FDPN-P2 network includes a self-attention processing unit; please refer again to Figure 2 Based on multiple feature fusion units and multiple RepC3 units in the FDPN-P2 network, scale features are optimized. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and This includes: scale features based on self-attention processing units. Self-attention processing is performed to obtain attention-enhancing features. Based on the first feature fusion unit, scale features are processed. , and attention enhancement features Feature processing is performed to obtain mesoscale features. ;Mesoscale features based on FDPN-P2 network The first upsampled feature is obtained by performing upsampling processing. The first upsampled feature Scale features The first stitched feature is obtained by performing a stitching process; and the mesoscale feature is then processed. The first convolutional feature is obtained by performing 3×3 convolution. , the first convolutional features Scale features The first spliced ​​feature is obtained by stitching together the features; the first spliced ​​feature is then fused and refined using the first RepC3 unit to obtain the first fused feature, and the second spliced ​​feature is then fused and refined using the second RepC3 unit to obtain the second fused feature; based on the second feature fusion unit, the mesoscale features are then processed... The first and second fusion features are fused to obtain multi-scale features. .

[0117] Multi-scale features based on FDPN-P2 network The second upsampled feature is obtained by performing upsampling processing. ; Second upsampling features based on FDPN-P2 network First upsampling feature The first fusion feature is then concatenated to obtain the third concatenated feature; the third RepC3 unit is then used to fuse and refine the third concatenated feature to obtain multi-scale features. .

[0118] Multi-scale features based on FDPN-P2 network Perform 3×3 convolution to obtain the second convolution feature. ;Based on FDPN-P2 network for second convolutional features First convolutional features The second fusion feature is then concatenated to obtain the fourth concatenated feature; the fourth RepC3 unit is then used to fuse and refine the fourth concatenated feature to obtain multi-scale features. .

[0119] Optionally, the scale features output by the backbone network can be... , , and As input to the hybrid encoder layer, the hybrid encoder layer includes an FDPN-P2 network, where scale features are first processed using the self-attention processing unit (AIFI) of the FDPN-P2 network. Self-attention processing is performed to obtain attention-enhancing features. .

[0120] scale features , and attention enhancement features The input is fed into the first feature fusion unit, i.e. Figure 2FocusFeature1, which represents the scale features at different scales. , and attention enhancement features Preliminary focused fusion was performed to obtain mesoscale features. and mesoscale features As the output of FocusFeature1, it is passed along multiple paths. Specifically, the first output path of FocusFeature1 handles mesoscale features. The first upsampled feature is obtained by performing upsampling processing. The first upsampled feature Scale features The features are concatenated to obtain the first concatenated feature, which is then used as the input to the first RepC3 unit. In the second output path of FocusFeature1, the mesoscale features are processed... The first convolutional feature is obtained by performing 3×3 convolution. , the first convolutional features Scale features The features are concatenated to obtain the second concatenated feature, which is then used as the input to the second RepC3 unit. The mesoscale features are then included in the third output path of FocusFeature1. It is directly fed as input into the second feature fusion unit, i.e. Figure 2 FocusFeature2 in the middle.

[0121] The first RepC3 unit, i.e. Figure 2 In 1RepC3, the first splicing feature is fused and refined to obtain the first fused feature. This first fused feature is passed along two paths and is used to participate in the FocusFeature2 feature fusion and splicing operations, respectively.

[0122] The second RepC3 unit, i.e. Figure 2 In 2RepC3, the second splicing feature is fused and refined to obtain the second fused feature. The second fused feature is passed along two paths: on the one hand, it is sent as input to FocusFeature2 to participate in multi-scale feature focusing, and on the other hand, it is used for subsequent feature splicing operations.

[0123] The second feature fusion unit (FocusFeature2) receives mesoscale features from FocusFeature1. The first fused feature of the first RepC3 unit and the second fused feature of the second RepC3 unit are combined and feature fusion processing is performed to obtain multi-scale features. The first output path of FocusFeature2 outputs multi-scale features. The second upsampled feature is obtained by performing upsampling processing. The second upsampled feature The first fused feature and the first upsampled feature output by the first RepC3 unit The features are concatenated to obtain a third concatenated feature, which is then used as the input to the third RepC3 unit. In the second output path of FocusFeature2, the multi-scale features are processed. Perform 3×3 convolution to obtain the second convolution feature. The second convolution feature The second fused feature and the first convolutional feature output by the second RepC3 unit The features are concatenated to obtain the fourth concatenated feature, which is then used as the input to the fourth RepC3 unit. The third output path of FocusFeature2 directly incorporates the multi-scale features. The output is one of the final multi-scale features.

[0124] The third RepC3 unit, i.e. Figure 2 In 3RepC3, the third concatenated feature is fused and refined to obtain multi-scale features. Furthermore, the fourth RepC3 unit, i.e. Figure 2 In 4RepC3, the fourth concatenated feature is fused and refined to obtain multi-scale features. .

[0125] In this embodiment, the FDPN-P2 network uses multiple feature fusion units and multiple RepC3 units to process scale features. and and scale features after self-attention processing By performing fusion processing, each scale feature can have detailed contextual information, and the features at different scales can be fused together, thus mitigating the problem of semantic loss.

[0126] In the previous exemplary embodiment, based on the first feature fusion unit in the FDPN-P2 network, scale features are... , and attention enhancement features Feature processing is performed to obtain mesoscale features. This includes: using a combination of downsampling, 1×1 convolution, and upsampling operations to optimize scale features. , and attention enhancement features Spatial resolution and channel number alignment are performed to obtain aligned features. These aligned features are then concatenated along the channel dimension to obtain aligned and concatenated features. Parallel depthwise convolutions are then performed on the aligned and concatenated features using multiple sets of convolutional kernels of different sizes. The outputs of each parallel depthwise convolution branch are then weighted and superimposed with the aligned and concatenated features to obtain superimposed features. Finally, pointwise convolutions are performed on the superimposed features to obtain mesoscale features. .

[0127] For example, Figure 4 This is a flowchart illustrating the feature fusion unit (i.e., the FocusFeature mentioned above). Taking the first feature fusion unit as an example, in the first feature fusion unit, downsampling, 1*1 convolution, and upsampling combined with 1*1 convolution are used to combine features of three different scales. , and attention enhancement features Spatial resolution and channel number alignment are performed to obtain aligned features. These aligned features are then concatenated along the channel dimension to obtain aligned and concatenated features. Multiple sets of large convolutional kernels (5, 7, 9, 11) of different sizes are used in parallel for depthwise convolution (DW Conv) processing on the aligned and concatenated features. Simultaneously, an identity direct connection is retained to directly reuse the aligned and concatenated features. That is, the outputs of each parallel DW Conv processing branch are weighted and superimposed with the aligned and concatenated features to obtain superimposed features, enabling weighted superposition of multi-scale context and original features. Pointwise convolution processing is performed on the superimposed features to complete information exchange and feature recombination between channels. Finally, mesoscale features are output through residual connections. Specifically, the stacked features after pointwise convolution are added element-wise to the aligned and concatenated features to obtain the mesoscale features. This allows us to obtain an enhanced feature map (mesoscale feature map) that contains both deep semantics and preserves shallow details. ).

[0128] Traditional RT-DETR models rely on CCFM to fuse features from different levels. However, during CCFM fusion, when deep features are upsampled and added to shallow features, the weak features of small targets are easily diluted by the stronger semantic information from larger targets. Therefore, this invention employs a feature fusion unit that focuses on extracting scale features from the backbone network. , and This allows each scale feature to have detailed contextual information and enables the fusion of scale features across different scales, thus mitigating the problem of semantic loss.

[0129] In an exemplary embodiment, the pre-training process of the traffic target detection model includes: detecting traffic targets in the training image based on a pre-acquired initial target detection model to obtain training prediction boxes; and determining the overlapping rectangular area between the training prediction boxes and the corresponding ground truth boxes in the training image based on the training prediction boxes and the ground truth boxes. and the area of ​​union ; where the area of ​​the overlapping rectangles The calculation formula is , , , and These represent the right, left, bottom, and top boundaries of the true bounding box, respectively. , , and These represent the right, left, bottom, and top boundaries of the training prediction box, respectively. This represents the smallest numerical value between the boundaries of the training predicted bounding box and the boundaries of the ground truth bounding box. The area of ​​the union of the training predicted bounding box and the ground truth bounding box represents the largest numerical value; The calculation formula is , and These represent the width and height of the training prediction box, respectively. and These represent the width and height of the actual bounding box, respectively. Scale factor; based on the area of ​​overlapping rectangles Area of ​​union The intersection-union ratio (IU) between the scaled-down training prediction boxes and the ground truth boxes is determined to be . Based on the cross-union ratio of the shrunk boxes, the SIoU loss value of the unshrunk training predicted boxes, and the cross-union ratio between the unshrunk training predicted boxes and the ground truth boxes, the objective loss function is determined as follows: , The SIoU loss value is the training prediction box without scaling. The cross-union ratio (CUP) is the ratio between the unscaled training predicted bounding boxes and the ground truth bounding boxes. The initial object detection model is optimized based on the objective loss function to obtain the traffic object detection model.

[0130] Traditional RT-DETR models use GIoU (Generalized Intersection over Union) as the bounding box regression loss. However, when dealing with complex roadside scenes, especially with distant small targets and severely occluded vehicles, the gradient of the traditional IoU mechanism is often small at low overlap rates, leading to slow convergence and insufficient localization accuracy in object detection models. Furthermore, current IoU-based bounding box regression methods mainly accelerate convergence by adding new loss terms, ignoring the inherent limitations of IoU loss itself. During object detection model training, distinguishing different regression samples and using auxiliary bounding boxes of different scales to calculate the loss can effectively accelerate the bounding box regression process. By analyzing the bounding box regression process and patterns, this paper proposes that using smaller auxiliary bounding boxes to calculate the loss during object detection model training has a beneficial effect on the regression of high IoU samples, improving the overall generalization ability of the model. Conversely, for low IoU samples, where the predicted and ground truth bounding boxes overlap less, a more significant gradient is provided by increasing the penalty for samples with poor prediction quality.

[0131] This invention proposes an Inner-SIoU (Inner Scalable Intersection over Union) loss function, which generates auxiliary bounding boxes of different scales based on a scale factor ratio to calculate the loss. By combining the angle-guided optimization of SIoU with the strict region constraints of Inner-IoU, faster convergence speed and higher accuracy are achieved. For example, traffic targets in training images are detected based on a pre-acquired initial object detection model to obtain training prediction boxes; based on the training prediction boxes and the corresponding ground truth boxes in the training images, the overlapping rectangular area between the training prediction boxes and the ground truth boxes is determined. for:

[0132]

[0133] In the above formula, , , and These represent the right, left, bottom, and top boundaries of the true bounding box, respectively. , , and These represent the right, left, bottom, and top boundaries of the training prediction box, respectively. This represents the smallest numerical value between the boundaries of the training predicted bounding box and the boundaries of the ground truth bounding box. This represents the largest value among the boundaries of the training predicted bounding box and the ground truth bounding box.

[0134] The area of ​​the union between the training predicted bounding box and the ground truth bounding box The calculation formula is as follows:

[0135]

[0136] in, and These represent the width and height of the training prediction box, respectively. and These represent the width and height of the actual bounding box, respectively. is the scale factor.

[0137] Based on the area of ​​overlapping rectangles Area of ​​union The intersection-union ratio (IU) between the scaled-down training predicted boxes and the ground truth boxes is determined as follows:

[0138]

[0139] The reduced-size box intersection-union ratio (IoU) is calculated by shrinking the effective regions of the training predicted boxes and the ground truth boxes, using the target core region as the calculation benchmark, and based on the intersection and union areas of the shrunken training predicted boxes and the ground truth boxes. Further, based on the reduced-size box IoU, the SIoU loss value of the unshrunken training predicted boxes, and the IoU between the unshrunken training predicted boxes and the ground truth boxes, the target loss function is determined as follows:

[0140]

[0141] In the formula, The SIoU loss value is the training prediction box without scaling. This represents the Cross-Union Ratio (SIoU) between the unscaled training predicted boxes and the ground truth boxes. Specifically, the SIoU loss value for the unscaled training predicted boxes refers to the SIoU value calculated based on the unscaled training predicted boxes and the ground truth boxes. The SIoU value is calculated using the Cross-Union Ratio, center point offset, and aspect ratio difference between the unscaled training predicted boxes and the ground truth boxes.

[0142] In this embodiment, the target loss function determined above can amplify the overlap weight of the effective area, accelerate the convergence of the traffic target detection model under high-precision threshold, and thus significantly improve the localization robustness of the traffic target detection model in nighttime and occluded scenarios.

[0143] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0144] In an exemplary embodiment, to demonstrate the advantages of the traffic target detection model (RTDETR-FP2 model) proposed in this invention in complex traffic scenarios, the traffic target detection model is compared with the traditional RT-DETR model, such as... Figure 5 The diagram shows a comparison of metrics between the traffic target detection model and the traditional RT-DETR model. In the diagram, P represents precision, R represents recall, and mAP represents mean precision.

[0145] Optionally, in comparing the traffic target detection model with the traditional RT-DETR model, the DAIR-V2X-I dataset is used. DAIR-V2X-I is the world's first large-scale dataset focusing on roadside multimodal perception, covering 10 kilometers of urban roads, 10 kilometers of highways, and 28 complex intersections. The core objective of the DAIR-V2X-I dataset is to simulate the perception capabilities of real roadside devices through data collection from multiple sensors deployed on road infrastructure. This DAIR-V2X-I dataset is specifically designed for environmental perception tasks of autonomous driving roadside infrastructure, providing rich sensor data and high-precision annotations. Furthermore, the dataset used in this comparative study consists of image data collected by roadside units (RSUs), containing eight target categories: cars, trucks, vans, buses, pedestrians, cyclists, motorcycles, and traffic cones. The final dataset contains 7,058 images, randomly divided into training, validation, and test sets in a 7:2:1 ratio.

[0146] like Figure 6 The image shows a schematic diagram of the metrics for each target category in the RT-DETR model, as follows: Figure 7The diagram shows the performance indicators for each target category in the traffic target detection model. It can be seen that compared to the traditional RT-DETR model, the traffic target detection model (RTDETR-FP2 model) proposed in this invention achieves significant breakthroughs in all key detection indicators. Experimental data shows that the precision, recall, mAP50, and mAP50-95 of this traffic target detection model have improved from 88.3%, 86.3%, 89.4%, and 67.9% to 89.2%, 87.2%, 90.5%, and 70.6%, respectively, demonstrating excellent overall detection performance. Particularly noteworthy is the significant advantage of this traffic target detection model in detecting small-sized targets. Specifically, the accuracy of the traffic target detection model is greatly improved in the detection of pedestrians, bicycles, and motorcycles. Taking pedestrian detection as an example, its mAP50 indicator jumped from the baseline of 89.8% to 92.4%. Under the more stringent mAP50-95 evaluation standard, the overall performance improved significantly from 56.7% to 62.9%. This improvement not only demonstrates the higher accuracy of the traffic object detection model in object bounding box regression but also reflects its stronger robustness and stability in complex scenarios. This performance enhancement is mainly attributed to the traffic object detection model's effective integration of low-level features containing rich details. This allows the traffic object detection model to maintain extremely high detection accuracy even when facing objects in real-world road scenarios that are difficult to identify due to their small scale or occlusion, such as distant vehicles and small pedestrians.

[0147] To verify the enhancement effects of the added FDPN network, FDPN-P2 network, BAM, and Inner-SIoU on the traffic object detection model, an ablation experiment was designed. By progressively introducing the FDPN network, FDPN-P2 network, BAM, and Inner-SIoU, the relevant experimental metrics (precision, recall, and mean average precision (mAP)) were validated. The ablation experiment results are shown in the table below.

[0148]

[0149] Ablation experiments show that the experimental metrics continuously improve with the addition and optimization of the traffic target detection model. Adding the FDPN network increases the model's precision, recall, mAP50, and mAP50-95 values ​​by 0.6%, 0.3%, 0.7%, and 1.4%, respectively, demonstrating that the FDPN network effectively diffuses features across various detection scales through a diffusion mechanism. Furthermore, fusing features from the P2 layer increases the model's precision, recall, mAP50, and mAP50-95 values ​​by 0.7%, 1.1%, 1%, and 2.6% compared to the base model, demonstrating that the P2 layer features effectively preserve rich detail and location information. By fusing P2 layer features with the FDPN network to form the FDPN-P2 network, the accuracy of small target detection in real-world road conditions can be significantly improved. Finally, the loss functions BAM and Inner-SIou were introduced. Compared with the original model, the model's precision, recall, mAP50, and mAP50-95 values ​​increased by 0.9%, 0.9%, 1.1%, and 2.7%, respectively, improving the accuracy of the traffic target detection model when multiple targets overlap. Ablation experiments fully demonstrated the rationality and stability of the module addition.

[0150] To further demonstrate the detection performance of the traffic target detection model proposed in this application, the model was compared with other models (such as YOLOv5, YOLOv8, YOLOv11, and RTDETR-r18) on the DAIR-V2X-I dataset. The experimental results are shown in the table below.

[0151]

[0152] As shown in the table above, the traffic target detection model proposed in this application demonstrates significant advantages in all performance indicators compared to various mainstream cutting-edge target detection models. Particularly noteworthy is the mAP50-95 metric, which reflects the model's comprehensive detection capability and high-precision positioning level; the proposed model leads all compared models with a score of 70.6%. Specifically, compared to the classic YOLOv5s, YOLOv8s, and the latest YOLOv11s models (63.8%, 65.0%, and 68.2%, respectively), the proposed model achieves significant improvements of 6.8%, 5.6%, and 2.4%, respectively. This result strongly demonstrates the superiority of the proposed traffic target detection model in feature extraction and multi-scale fusion, indicating that it possesses stronger generalization ability and robustness than existing mainstream lightweight models when facing detection tasks of varying difficulty.

[0153] Notably, the proposed traffic target detection model achieves the most significant improvement of 2.7% on the mAP50-95 metric, which has the most stringent requirements for bounding box regression accuracy. This substantial increase has important physical significance: it means that the traffic target detection model can not only "detect" targets but also provide more closely aligned predicted bounding boxes that fit the target edges. In real-world complex road scenarios, this high-precision localization capability is crucial for identifying small, distant targets and partially occluded targets. This fully validates that the fusion module of the introduced FDPN and P2 layers effectively alleviates the feature ambiguity problem and significantly enhances the stability and reliability of the algorithm under extreme road conditions.

[0154] Furthermore, this traffic target detection model still demonstrates superior performance when compared across scales with the RTDETR-r50, which has a much larger number of parameters. Despite the r50's deeper network structure, this traffic target detection model, thanks to its efficient feature pyramid design, outperforms r50 by 70.6% on the mAP50-95 metric (69.5%). This indicates that optimizing the feature fusion mechanism to address the feature sparsity problem of small targets is more effective than simply increasing network depth. Simultaneously, its accuracy of 89.2% ranks first among all compared models, demonstrating that while significantly improving detection capabilities, it effectively controls the risk of false detections, achieving a dual breakthrough in accuracy and robustness.

[0155] Based on the same inventive concept, this application also provides a target detection device for complex traffic scenarios to implement the target detection method for complex traffic scenarios described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations of one or more embodiments of the target detection device for complex traffic scenarios provided below can be found in the limitations of the target detection method for complex traffic scenarios described above, and will not be repeated here.

[0156] In one exemplary embodiment, such as Figure 8 As shown, a target detection device for complex traffic scenarios is provided, including: a target detection module 802, wherein:

[0157] The target detection module 802 is used to determine the detection results of traffic targets in the image to be tested based on a pre-trained traffic target detection model. The traffic target detection model includes a backbone network, a hybrid encoder layer, a perceptual query selection layer, and a detection head layer. The hybrid encoder layer includes an FDPN-P2 network. The process of determining the detection results of traffic targets in the image to be tested based on the traffic target detection model includes: performing convolution and pooling processing on the image to be tested based on the backbone network to obtain the original features; and inputting the original features into multiple residual blocks of the backbone network to obtain scale features. , , and Based on multiple feature fusion units and multiple RepC3 units in the FDPN-P2 network, scale features are optimized. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and In the FDPN-P2 network, the multi-scale features after upsampling are... and scale features Feature concatenation and feature fusion are performed to obtain multi-scale features. Based on the perceptual query selection layer for multi-scale features , , and Multiple tokens are obtained by performing feature flattening, and these tokens are then processed based on the perception query selection layer and the detection head layer to obtain the detection results of traffic targets.

[0158] In an exemplary embodiment, the residual block includes a main path and shortcut branches; the object detection module 802 is further configured to, for each residual block, sequentially process the original features based on a first feature convolutional layer and a second feature convolutional layer in the main path of that residual block to obtain input features. And based on the main path BAM, the input features are analyzed. Feature enhancement processing is performed to obtain refined features. Based on the shortcut branch of the residual block, determine the step size and channel dimension of the residual block and the refinement features. Whether the step size and channel dimension match; when it is determined that the step size and channel dimension of the residual block match the refinement feature. When the step size and channel dimension do not match, in the shortcut branch, average pooling and channel alignment are performed on the original features to obtain aligned original features; then the aligned original features are combined with the refined features. Element-wise addition is performed to obtain a scale feature; when the step size and channel dimension of the residual block are determined, the refinement feature is... When matching the step size and channel dimension, the original features are compared with the refined features. By performing element-wise addition, a scale feature is obtained.

[0159] In an exemplary embodiment, the target detection module 802 is further configured to process the input features. The input features are obtained by performing average pooling, multilayer sensing, and normalization. Corresponding channel attention features Channel attention features The calculation formula is ,in, Indicates batch normalization. This represents a multilayer perceptron. Indicates average pooling. and These represent the weights of the dimensionality reduction layer and the weights of the dimensionality increase layer, respectively. and These represent the bias parameters of the dimensionality reduction layer and the bias parameters of the dimensionality increase layer, respectively; for the input features Convolution and normalization are performed to obtain the input features. Corresponding spatial attention features Spatial attention characteristics The calculation formula is ,in, This represents a 1×1 convolutional kernel located in the first convolutional layer. This represents a 3×3 convolutional kernel located in the second convolutional layer. This represents a 3×3 convolutional kernel located in the third convolutional layer. This represents a 1×1 convolutional kernel located in the fourth convolutional layer; channel attention features. Spatial attention features Weighted fusion processing is performed to obtain a 3D attention map. 3D attention map The calculation formula is ,in, It is the sigmoid function; based on a 3D attention map. For input features After processing, the refined features are obtained. .

[0160] In one exemplary embodiment, the FDPN-P2 network includes a self-attention processing unit; the target detection module 802 is further configured to perform scale feature analysis based on the self-attention processing unit. Self-attention processing is performed to obtain attention-enhancing features. Based on the first feature fusion unit, scale features are processed. , and attention enhancement features Feature processing is performed to obtain mesoscale features. ;Mesoscale features based on FDPN-P2 network The first upsampled feature is obtained by performing upsampling processing. The first upsampled feature Scale features The first stitched feature is obtained by performing a stitching process; and the mesoscale feature is then processed. The first convolutional feature is obtained by performing 3×3 convolution. , the first convolutional features Scale features The first spliced ​​feature is obtained by stitching together the features; the first spliced ​​feature is then fused and refined using the first RepC3 unit to obtain the first fused feature, and the second spliced ​​feature is then fused and refined using the second RepC3 unit to obtain the second fused feature; based on the second feature fusion unit, the mesoscale features are then processed... The first and second fusion features are fused to obtain multi-scale features. .

[0161] In an exemplary embodiment, the target detection module 802 is further configured to perform multi-scale feature analysis based on the FDPN-P2 network. The second upsampled feature is obtained by performing upsampling processing. ; Second upsampling features based on FDPN-P2 network First upsampling feature The first fusion feature is then concatenated to obtain the third concatenated feature; the third RepC3 unit is then used to fuse and refine the third concatenated feature to obtain multi-scale features. .

[0162] In an exemplary embodiment, the target detection module 802 is further configured to perform multi-scale feature analysis based on the FDPN-P2 network. Perform 3×3 convolution to obtain the second convolution feature. ;Based on FDPN-P2 network for second convolutional features First convolutional features The second fusion feature is then concatenated to obtain the fourth concatenated feature; the fourth RepC3 unit is then used to fuse and refine the fourth concatenated feature to obtain multi-scale features. .

[0163] In an exemplary embodiment, the target detection module 802 is further configured to employ a combination of downsampling, 1×1 convolution, and upsampling operations to optimize scale features. , and attention enhancement features Spatial resolution and channel number alignment are performed to obtain aligned features. These aligned features are then concatenated along the channel dimension to obtain aligned and concatenated features. Parallel depthwise convolutions are then performed on the aligned and concatenated features using multiple sets of convolutional kernels of different sizes. The outputs of each parallel depthwise convolution branch are then weighted and superimposed with the aligned and concatenated features to obtain superimposed features. Finally, pointwise convolutions are performed on the superimposed features, and mesoscale features are output through residual connections. .

[0164] In one exemplary embodiment, the target detection device in complex traffic scenarios further includes:

[0165] The model training module is used to detect traffic targets in training images based on a pre-acquired initial object detection model, obtaining training prediction boxes; and based on the training prediction boxes and the corresponding ground truth boxes in the training images, determining the area of ​​the overlapping rectangle between the training prediction boxes and the ground truth boxes. and the area of ​​union ; where the area of ​​the overlapping rectangles The calculation formula is , , , and These represent the right, left, bottom, and top boundaries of the true bounding box, respectively. , , and These represent the right, left, bottom, and top boundaries of the training prediction box, respectively. This represents the smallest numerical value between the boundaries of the training predicted bounding box and the boundaries of the ground truth bounding box. The area of ​​the union of the training predicted bounding box and the ground truth bounding box represents the largest numerical value; The calculation formula is , and These represent the width and height of the training prediction box, respectively. and These represent the width and height of the actual bounding box, respectively. Scale factor; based on the area of ​​overlapping rectangles Area of ​​union The intersection-union ratio (IU) between the scaled-down training prediction boxes and the ground truth boxes is determined to be . Based on the cross-union ratio of the shrunk boxes, the SIoU loss value of the unshrunk training predicted boxes, and the cross-union ratio between the unshrunk training predicted boxes and the ground truth boxes, the objective loss function is determined as follows: , The SIoU loss value is the training prediction box without scaling. The cross-union ratio (CUP) is the ratio between the unscaled training predicted bounding boxes and the ground truth bounding boxes. The initial object detection model is optimized based on the objective loss function to obtain the traffic object detection model.

[0166] The various modules in the target detection device for the aforementioned complex traffic scenarios can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0167] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0168] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0169] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0170] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0171] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0172] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0173] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A target detection method in complex traffic scenarios, characterized in that, The method includes: The detection results of traffic targets in the image to be tested are determined based on a pre-trained traffic target detection model. The traffic target detection model includes a backbone network, a hybrid encoder layer, a perceptual query selection layer, and a detection head layer. The hybrid encoder layer includes an FDPN-P2 network. The process of determining the detection results of traffic targets in the image to be tested based on the traffic target detection model includes: The backbone network performs convolution and pooling processes on the image under test to obtain the original features. These original features are then input into multiple residual blocks of the backbone network to obtain scale features. , , and ; Based on multiple feature fusion units and multiple RepC3 units in the FDPN-P2 network, scale features are optimized. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and ; In the FDPN-P2 network, the upsampled multi-scale features and scale features Feature concatenation and feature fusion are performed to obtain multi-scale features. ; Based on the perceptual query selection layer, multi-scale features , , and Multiple tokens are obtained by performing feature flattening, and these tokens are then processed based on the perception query selection layer and the detection head layer to obtain the detection results of traffic targets.

2. The method according to claim 1, characterized in that, The residual block includes the main path and shortcut branches; the original features are input into multiple residual blocks of the backbone network to obtain scale features. , , and ,include: For each residual block, in the main path of that residual block, the original features are sequentially processed based on the first feature convolutional layer and the second feature convolutional layer to obtain the input features. And based on the BAM of the main path, the input features are... Feature enhancement processing is performed to obtain refined features. ; Based on the shortcut branch of the residual block, determine the step size and channel dimension of the residual block in relation to the refined features. Whether the step size and channel dimension match; When the step size and channel dimension of the residual block are determined to be in relation to the refinement feature... When the step size and channel dimension do not match, in the shortcut branch, the original features are subjected to average pooling and channel alignment to obtain aligned original features; and the aligned original features are then compared with the refined features. By performing element-wise addition, a scale feature is obtained; When the step size and channel dimension of the residual block are determined to be in relation to the refinement feature... When matching the step size and channel dimension, the original feature is compared with the refined feature. By performing element-wise addition, a scale feature is obtained.

3. The method according to claim 2, characterized in that, The BAM based on the main path applies to the input features. Feature enhancement processing is performed to obtain refined features. ,include: For the input features The input features are obtained by performing average pooling, multilayer sensing, and normalization. Corresponding channel attention features The channel attention features The calculation formula is ,in, Indicates batch normalization. This represents a multilayer perceptron. Indicates average pooling. and These represent the weights of the dimensionality reduction layer and the weights of the dimensionality increase layer, respectively. and These represent the bias parameters of the dimensionality reduction layer and the bias parameters of the dimensionality increase layer, respectively. For the input features The input features are obtained by performing convolution and normalization. Corresponding spatial attention features The spatial attention features The calculation formula is ,in, This represents a 1×1 convolutional kernel located in the first convolutional layer. This represents a 3×3 convolutional kernel located in the second convolutional layer. This represents a 3×3 convolutional kernel located in the third convolutional layer. This represents a 1×1 convolutional kernel located in the fourth convolutional layer; Channel attention features and the spatial attention features Weighted fusion processing is performed to obtain a 3D attention map. The three-dimensional attention map The calculation formula is ,in, It is the sigmoid function; Based on the three-dimensional attention map For the input features After processing, the refined features are obtained. .

4. The method according to claim 1, characterized in that, The FDPN-P2 network includes a self-attention processing unit; the multiple feature fusion units and multiple RepC3 units based on the FDPN-P2 network are used for scale features. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and ,include: Based on the scale features of the self-attention processing unit Self-attention processing is performed to obtain attention-enhancing features. ; Based on the first feature fusion unit, scale features , and attention enhancement features Feature processing is performed to obtain mesoscale features. ; Based on the FDPN-P2 network, the mesoscale features are analyzed. The first upsampled feature is obtained by performing upsampling processing. The first upsampled feature With the scale features Perform stitching processing to obtain the first stitched feature; and process the mesoscale feature. The first convolutional feature is obtained by performing 3×3 convolution. The first convolutional feature With the scale features The splicing process is performed to obtain the second splicing feature; The first splicing feature is obtained by fusing and refining the first splicing feature using the first RepC3 unit, and the second splicing feature is obtained by fusing and refining the second splicing feature using the second RepC3 unit. Based on the second feature fusion unit, the mesoscale features The first and second fusion features are fused to obtain multi-scale features. .

5. The method according to claim 4, characterized in that, The method further includes: Based on the FDPN-P2 network, the multi-scale features are analyzed. The second upsampled feature is obtained by performing upsampling processing. ; Second upsampling features based on FDPN-P2 network First upsampling feature The first fusion feature is then spliced ​​together to obtain the third spliced ​​feature; The third splicing feature is fused and refined using the third RepC3 unit to obtain multi-scale features. .

6. The method according to claim 4, characterized in that, The method further includes: Based on the FDPN-P2 network, the multi-scale features are analyzed. Perform 3×3 convolution to obtain the second convolution feature. ; The second convolutional feature is based on the FDPN-P2 network. First convolutional features The second fusion feature is then spliced ​​together to obtain the fourth splicing feature; The fourth RepC3 unit is used to fuse and refine the fourth spliced ​​feature to obtain multi-scale features. .

7. The method according to claim 4, characterized in that, The first feature fusion unit in the FDPN-P2 network performs scale features. , and attention enhancement features Feature processing is performed to obtain mesoscale features. ,include: A combination of downsampling, 1×1 convolution, and upsampling operations are used to process scale features. , and attention enhancement features Spatial resolution and channel number alignment processing is performed to obtain aligned features, and the aligned features are then stitched together along the channel dimension to obtain aligned and stitched features. The aligned and stitched features are processed by parallel depth convolution based on multiple sets of convolution kernels of different sizes, and the output of each parallel depth convolution processing branch is weighted and superimposed with the aligned and stitched features to obtain the superimposed features. The superimposed features are subjected to pointwise convolution processing, and the mesoscale features are output through residual connections. .

8. The method according to claim 1, characterized in that, The pre-training process of the traffic target detection model includes: Traffic targets in training images are detected based on a pre-acquired initial target detection model to obtain training prediction boxes; Based on the training prediction bounding boxes and the corresponding ground truth bounding boxes in the training images, determine the area of ​​the overlapping rectangle between the training prediction bounding boxes and the ground truth bounding boxes. and the area of ​​union Wherein, the area of ​​the overlapping rectangle The calculation formula is , , , and These represent the right, left, bottom, and top boundaries of the true bounding box, respectively. , , and These represent the right, left, bottom, and top boundaries of the training prediction box, respectively. This represents the smallest numerical value between the boundaries of the training predicted bounding box and the boundaries of the ground truth bounding box. The union area represents the largest value between the boundaries of the training predicted bounding box and the boundaries of the ground truth bounding box; The calculation formula is , and These represent the width and height of the training prediction box, respectively. and These represent the width and height of the actual bounding box, respectively. Scale factor; Based on the area of ​​the overlapping rectangle and the area of ​​the union The intersection-union ratio (IU) between the scaled-down training prediction boxes and the ground truth boxes is determined to be . ; Based on the cross-union ratio of the reduced boxes, the SIoU loss value of the unreduced training predicted boxes, and the cross-union ratio between the unreduced training predicted boxes and the ground truth boxes, the target loss function is determined as follows: , The SIoU loss value is the training prediction box without scaling. The intersection-union ratio (IU) between the unscaled training predicted boxes and the ground truth boxes; The initial target detection model is optimized based on the target loss function to obtain a traffic target detection model.

9. A target detection device for complex traffic scenarios, characterized in that, The device includes: The target detection module is used to determine the detection results of traffic targets in the image to be tested based on a pre-trained traffic target detection model. The traffic target detection model includes a backbone network, a hybrid encoder layer, a perceptual query selection layer, and a detection head layer. The hybrid encoder layer includes an FDPN-P2 network. The process of determining the detection results of traffic targets in the image to be tested based on the traffic target detection model includes: The backbone network performs convolution and pooling processes on the image under test to obtain the original features. These original features are then input into multiple residual blocks of the backbone network to obtain scale features. , , and ; Based on multiple feature fusion units and multiple RepC3 units in the FDPN-P2 network, scale features are optimized. and and scale features after self-attention processing The fusion process is performed to obtain multi-scale features. , and ; In the FDPN-P2 network, the upsampled multi-scale features and scale features Feature concatenation and feature fusion are performed to obtain multi-scale features. ; Based on the perceptual query selection layer, multi-scale features , , and Multiple tokens are obtained by performing feature flattening, and these tokens are then processed based on the perception query selection layer and the detection head layer to obtain the detection results of traffic targets.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

Images