Vehicle detection method and device based on millimeter wave radar and vision fusion

Abstract

The invention discloses a vehicle detection method and device based on millimeter wave radar and vision fusion, and relates to the technical field of target detection. Filtering and screening the acquired millimeter wave radar data by setting a screening condition to obtain effective radar data, and constructing a data set in combination with the acquired image data of the vehicle; based on a D-CRFNet network, features are extracted from the data set, data features are obtained, and the data features comprise image features and sequence features; a double-layer encoder is arranged, the encoder carries out position encoding on the data features and carries out fusion, and fusion features are obtained; processing the fusion features by using an attention mechanism to obtain enhanced fusion features; setting a decoder to decode the enhanced fusion features to obtain a target boundary and a column tag; and screening and classifying the obtained target boundary and column tag by setting a screening condition and non-maximum suppression to obtain a vehicle target detection result.

Description

Technical Field

[0001] The present invention relates to the field of target detection technology, and in particular to a vehicle detection method and device based on millimeter-wave radar and vision fusion. Background Art

[0002] At present, the main methods for vehicle detection on highways include video speed measurement, electromagnetic induction coil speed measurement and laser detection. Digital and image methods are used to take pictures of passing vehicles, record the time and distance when the vehicle enters and leaves the detection area, and then derive the speed of the vehicle in this area to achieve the goal of vehicle speed detection. However, the collected vehicle pictures are high-resolution images or large image processing. When processing high-resolution images or large images, the memory consumption is too large, and in this case, the ability to capture the global context is limited, resulting in insufficient accuracy of the detection results. Summary of the invention

[0003] In order to overcome the shortcomings of the above-mentioned prior art that when processing high-resolution images or large images, the memory consumption is too large and the ability to capture the global context is limited, resulting in insufficient accuracy of the detection results, the main purpose of the present invention is to provide a vehicle detection method and device based on millimeter-wave radar and vision fusion.

[0004] To achieve the above object, the present invention adopts the following technical solution, a vehicle detection method based on millimeter wave radar and vision fusion, comprising the following steps:

[0005] The acquired millimeter-wave radar data is filtered by setting filtering conditions to obtain valid radar data, and the data set is constructed by combining the acquired vehicle image data;

[0006] Based on the D-CRFNet network, features are extracted from the data set to obtain data features, wherein the data features include image features and radar sequence features;

[0007] A double-layer encoder is set to encode the data features, and the image features and radar sequence features are respectively input into different layers of the double-layer encoder, and are respectively position-encoded and fused by weighted splicing to obtain fused features; the fused features are processed by an attention mechanism to obtain enhanced fused features;

[0008] Set up a decoder to decode the enhanced fusion features to obtain the target boundary and column labels;

[0009] By setting filtering conditions and non-maximum suppression, the obtained target boundaries and column labels are filtered and classified to obtain the vehicle target detection results.

[0010] The encoder processes the data features by:

[0011] The effective radar data is converted into a sequence, and the corresponding position code is generated according to the detection distance corresponding to the effective radar data to obtain the radar code data

[0012] Position-encode the image features to obtain image encoding data, and fuse them with the radar encoding data to obtain fusion features;

[0013] The attention mechanism is a multi-head self-attention mechanism, which processes the fusion features formed by radar and image features to obtain enhanced fusion features;

[0014] The enhanced fusion features are further processed using multi-layer perception to obtain the output of the encoder.

[0015] The obtaining of radar coded data comprises the following steps:

[0016] Based on the valid radar data, the location information of each radar point is determined, including the x, y, z coordinates and timestamp.

[0017] Project the three-dimensional position information of the radar point onto the image plane to obtain the corresponding position of each radar point in the image;

[0018] Determine the corresponding patch size based on the distance from the radar point to the radar sensor;

[0019] Get each patch on the image plane and arrange them into a sequence according to the order of each patch in the image;

[0020] The features of each patch are extracted and combined with its position information in the image to generate a position coding sequence as radar coding data.

[0021] The step of obtaining the image coding data comprises the following steps:

[0022] Obtaining the position of each pixel in the image feature;

[0023] The position of each pixel is multiplied by two different sine functions with a period of 2Π, and two results are obtained as image coding data; one result is a sine function with a period of 2Π, and the other result is a cosine function with a period of 2Π.

[0024] The double-layer encoder includes a first-layer encoder and a second-layer encoder. The first-layer encoder uses the radar sequence as a query and the image feature sequence as a key and a value; the second-layer encoder uses the image feature sequence as a query and the radar sequence as a key and a value.

[0025] The decoding process includes:

[0026] compressing the image features to obtain compressed image features;

[0027] Project the position code of the radar feature sequence onto the image feature map to generate a region of interest;

[0028] Use multi-head cross attention mechanism to extract features from the region of interest and lock the target position;

[0029] A multi-head self-attention mechanism is used to accurately detect the target and obtain the target boundary and column label as the decoder output.

[0030] The number of the object boundaries is greater than the number of actual objects in the image.

[0031] A vehicle detection system based on millimeter wave radar and vision fusion, comprising:

[0032] The data processing module is used to filter and screen the acquired millimeter-wave radar data by setting screening conditions to obtain effective radar data, and to construct a data set in combination with the acquired vehicle image data; based on the D-CRFNet network, the data set is extracted to obtain data features, which include image features and sequence features;

[0033] The encoding module is used to set a double-layer encoder to encode the data features, input the image features and radar sequence features into different layers of the double-layer encoder respectively, perform position encoding respectively, and then fuse them through weighted splicing to obtain fusion features; use the attention mechanism to process the fusion features to obtain enhanced fusion features;

[0034] The decoding module sets a decoder to decode the enhanced fusion features to obtain the target boundary and column labels;

[0035] The detection result acquisition module is used to filter and classify the obtained target boundaries and column labels by setting filtering conditions and non-maximum suppression to obtain vehicle target detection results.

[0036] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention uses D-CRFNet to extract the features of the two sensors, and through the bilateral encoder-decoder setting, it can effectively extract and fuse image and radar features, improve detection efficiency, and use the self-attention mechanism to capture global features and the relationship between different modalities. The spatial and contextual characteristics of cross-modalities are fully exploited to make up for the limitations of a single sensor, which can improve classification accuracy, and the decoder based on the multi-head attention mechanism can quickly lock the target position and generate accurate bounding boxes and category labels. Through the processing of FFN, the output result is fixed in size, which solves the uncertainty caused by the number of dynamic targets, effectively utilizes the advantages of the two sensors, and improves the overall robustness of target detection.

[0037] The present invention has the function of multi-scale analysis, can process and analyze images of multiple scales, capture features from local and global perspectives, and ensure that key details are retained. This method is particularly suitable for high-resolution images, effectively avoiding the loss of key details that may be caused by low-resolution images, thereby improving classification accuracy. By fusing high-resolution and low-resolution images and using graph pooling technology, the semantic parsing ability and processing efficiency of the model are further enhanced. This method not only retains the details in high-resolution images, but also utilizes the global information of low-resolution images, thereby improving the robustness and accuracy of the model. Moreover, in limited data scenarios, by fusing image information of multiple scales and adopting graph pooling technology, the risk of overfitting is effectively reduced and the generalization ability of the model is improved. Even in the case of limited labeled data, it can provide doctors with accurate representation of nodule size, shape and position, thereby improving diagnosis and treatment efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The drawings described herein are used to provide further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute improper limitations on the present application.

[0039] Figure 1 is a schematic diagram of radar data visualization in the present invention;

[0040] Figure 2 It is a schematic diagram of the corresponding position coding of the projection area in the present invention;

[0041] Figure 3 It is a schematic diagram of the flow of the encoder in the present invention;

[0042] Figure 4 is a schematic diagram of the decoder architecture of the present invention;

[0043] Figure 5 It is a flow chart of the loss function in the present invention.

[0044] Figure 6It is a schematic diagram of the process structure of the present invention. DETAILED DESCRIPTION

[0045] At present, the main methods for vehicle detection on highways include video speed measurement, electromagnetic induction coil speed measurement and laser detection. Digital and image methods are used to take photos of vehicles passing by, record the time and distance of vehicles entering and leaving the detection area, and thus obtain the speed of the vehicle in the area to achieve the goal of vehicle speed detection. However, this method is easily interfered by factors such as haze, rainy weather, and changes in the earth's magnetic field, resulting in low measurement accuracy. The present invention aims to provide a detection algorithm based on 4D millimeter-wave radar and visual fusion to improve the detection accuracy of vehicles.

[0046] Obstacles detected by millimeter-wave radar, such as stationary vehicles, street lights, people, and trees on the side of the road, have a significant impact on the vehicle under test and should be screened out. In addition, useless information such as turns or sudden braking during driving should also be screened out.

[0047] Useless information is filtered out by setting filtering conditions, and only the remaining target data is visualized and then projected onto radar monitoring to obtain relevant speed information. Figure 1 .

[0048] Millimeter-wave radar can accurately estimate the speed of an object in bad weather, but due to the sparse point cloud, it is impossible to complete accurate modeling, which easily leads to errors. Fusion of millimeter-wave radar and vision can achieve comprehensive perception of the environment, unaffected by weather, and improve detection accuracy. Specifically include the following:

[0049] The acquired millimeter-wave radar data is filtered by setting filtering conditions to obtain valid radar data, and the data set is constructed by combining the acquired vehicle image data;

[0050] Based on the D-CRFNet network, features are extracted from the data set to obtain data features, where the data features include image features and sequence features;

[0051] An encoder is set, wherein the encoder performs position encoding and fusion on the data features to obtain fusion features; and the fusion features are processed using an attention mechanism to obtain enhanced fusion features;

[0052] Set up a decoder to decode the enhanced fusion features to obtain the target boundary and column labels;

[0053] By setting filtering conditions and non-maximum suppression, the obtained target boundaries and column labels are filtered and classified to obtain the vehicle target detection results.

[0054] Embodiment 1:

[0055] In this embodiment, the D-CRFNet algorithm based on the attention mechanism uses the Transformer model to process radar and image features, effectively utilizes the spatial and contextual characteristics of the camera and radar, and combines self-attention and cross-attention mechanisms to improve the robustness and robustness of target detection. Using radar data in multimodality as feature assistance, a Transformer model capable of processing image and radar multimodal data is designed. Based on the proposed D-CRFNet algorithm extracting radar and image data features, the encoder and decoder of the Transformer are used for processing respectively. The radar and image features output by the D-CRFNet network backbone are used as the input of the model encoder. The encoder is used to create the position encoding of the radar and image and perform self-attention calculations, and the output of the encoder and the radar encoding sequence are respectively input to the decoder. In the decoder, the radar and image fusion point positions are self-attention and cross-attention calculations are performed, and finally the anchor frame and classification results are obtained.

[0056] First, the position encoding of the radar point needs to be determined. Therefore, the patch corresponding to the radar needs to be generated before the cross-attention processing. The Transformer model will partition the feature map into blocks and convert it into a sequence before processing the feature map. Patch refers to the element in the sequence, and each patch corresponds to a unique pixel position in the image. The patch is not set to a fixed size, but the size of the patch is designed as a function proportional to the radar detection distance. Because the millimeter-wave radar points are relatively sparse, five consecutive frames of data are used as sampling points. The pixel feature values ​​in the projection area are converted into the radar position encoding sequence to facilitate the subsequent processing of the encoder and decoder.

[0057] After obtaining the position encoding of the radar point sequence, the image feature map extracted by the backbone network is also position encoded, so that the Transformer model can find each pixel position when processing the image. For each position, it is multiplied by two different sine functions with a period of 2Π, and two results are obtained respectively. The first result is a sine function with a period of 2Π, and the second result is a cosine function with a period of 2Π. In this way, by calculating the function values ​​of the sine and cosine functions with different periods, the unique encoding of each position is obtained. The radar and image features are converted into normalized vectors that can be processed by the Transformer through LN, and the normalized vectors are input into the multi-head self-attention layer (Multihead Self-Attention, MSA) for processing. Self-attention calculates the eigenvalues ​​of feature vectors of the same type, and the multi-head self-attention operation performs multiple types of feature processing on the radar and image feature vectors. Finally, these multiple types of feature processing results are spliced ​​together, so that the encoder captures richer feature information. The radar and image feature vectors after MSA processing are fused and input into the multi-layer perceptron layer for processing. The sequence processed by MLP is the output of the encoder.

[0058] Similar to a, Figure b shows the second-layer encoder, which uses the image feature sequence as the query and the radar sequence as the key and value, achieving complementarity with the first-layer feature information.

[0059] The decoder uses radar data to generate regions of interest for target detection. First, the decoder compresses the image features along the vertical dimension. Then, the position encoding of the radar feature sequence is projected onto the image feature map plane at the corresponding position to generate a region of interest. Multi-head cross attention and self-attention are then calculated for the region to quickly lock the target position and accurately detect the target, avoiding the loss of target information. The number of input elements to the decoder is N. After FFN processing, FFN will eventually output N bounding boxes and category labels. But usually, the number N is much larger than the number of actual targets in the image. Special category labels are required to represent these undetected targets. When processing bounding boxes, the size and position of the bounding boxes need to be carefully considered to ensure their accuracy and rationality.

[0060] After the encoder-decoder process, a set of predictions of fixed size N will be output. Assuming that the radar projection onto the image generates m projection points, Nm random predictions will be randomly generated based on m regions of interest to complete the N predictions. Through the decoder process, N prediction results can be generated, which contain a binary (c, box) consisting of the target category and location, where c represents the category of the object and box represents the location of the target detection box. By applying the Hungarian algorithm, the algorithm can match the elements of the prediction set with the elements of the true set one by one, so as to calculate the difference between each prediction result and the true result and minimize the total loss. The calculation process of the bipartite graph matching loss function based on the Hungarian algorithm includes the target category to be predicted and the labeled target category. Through the calculation of the above loss function, the target to be predicted will obtain the category information c and location information box of multiple anchor boxes to be predicted. If there are less than n target categories in the labeled target category, null values ​​will be used to fill them. Finally, all the targets to be predicted and the labeled targets are matched one by one to calculate the loss and obtain the minimum loss.

[0061] The D-CRFNet algorithm based on the attention mechanism can effectively combine the images and radar data in the nuScenes dataset, thereby realizing the effective collection and analysis of multi-sensor data. Then, by using the model based on the attention mechanism, information from different data sources can be processed in target detection. Finally, the weight model file is used to verify a variety of complex road sections and environments. The results show that the D-CRFNet algorithm based on the attention mechanism has good performance in different environments.

[0062] In complex vehicle-road scenarios, there are problems such as inaccurate target detection due to partial occlusion of vehicles and pedestrians and small targets. The attention mechanism is introduced and the D-CRFNet algorithm based on the attention mechanism is proposed. Global attention is introduced to learn the global context information of the fusion point, and multi-scale features are added to solve the problem of partial occlusion of vehicles and pedestrians in the existing sensor fusion target detection algorithm in complex and dense scenes, and the recognition number of the main categories is increased, improving the robustness of fusion detection.

[0063] Embodiment 2:

[0064] The D-CRFNet algorithm in the present invention is a network for image segmentation, which combines a deep convolutional neural network (CNN) and a conditional random field (CRF). In this embodiment, the deep convolutional neural network (CNN) is selected as a group of VGG networks consisting of 5 convolutional layers, 3 fully connected layers and 1 Softmax output layer, which can effectively extract features in the image, thereby achieving accurate recognition and classification of the image. The first step of the network is to use 5 convolutional layers to capture the features of the image, and each convolutional layer is composed of multiple convolution kernels. Between every two convolutional layers, the VGG network uses a maximum pooling operation to reduce the size and number of parameters of the image to reduce the computational cost. The VGG network uses 3 fully connected layers to further process the features. The function of these layers is to compress the feature vectors of the convolutional layer and map them to the final output category. The activation function uses the ReLU function, which can accelerate training and avoid gradient disappearance. Finally, the Softmax function is used to divide the image into multiple types, and the probability distribution of each type is calculated in a normalized manner to determine the final classification result.

[0065] When extracting features, offsets are added to the original convolution kernel to increase the size of the receptive field and the spatial sampling position in the module. The offsets are learned by the CNN network channel with the same resolution. Each point of the convolution kernel is moved by a learnable displacement, and the spatial range and expansion rate of the convolution kernel and the current convolution layer can also be learned. The convolution kernel is used to sample on the feature map and identify the weight value of each position. The convolution kernel can define the expansion coefficient and size of the receptive field and output the feature value of each position.

[0066] The decoder uses radar data to generate a region of interest for target detection. First, the decoder compresses the image features along the vertical dimension. Then, the position encoding of the radar feature sequence is projected onto the image feature map plane at the corresponding position to generate a region of interest. Multi-head cross-attention and self-attention are then calculated on the region to quickly lock the target position and accurately detect the target, avoiding the loss of target information.

[0067] When accurately detecting targets, the decoder also follows the standard architecture of Transformer. The input is the sequence output by the encoder and N ObjectQuery, referred to as Q. Q determines the category and position of the final output of the decoder. In addition, Q also represents the key detection area of ​​target detection. In the decoder, the position and category of the Q query are generally randomly generated. This random generation method greatly increases the time for model training and reduces the efficiency of target detection. Therefore, radar feature points and Q areas are used for weighted average input to the decoder, so that the target detection area of ​​the decoder focuses on the area with radar points to achieve accurate detection. Finally, the decoder transmits the output encoding sequence to the feedforward network FFN, and detects the target according to the class label and bounding box set by FFN.

[0068] Example 3

[0069] Based on Example 1, the present invention uses the nuScenes dataset, which contains multi-frame radar point cloud data and synchronized image data. The dataset includes 25 scenes, including scenes with different weather, lighting, and dense traffic. The target class includes 7 main categories, such as cars, pedestrians, bicycles, traffic signs, etc.

[0070] The radar point cloud is projected onto the image plane to generate a position encoding sequence. Continuously sample 5 frames of radar points to enhance the stability of spatial and temporal features. The generated point cloud sequence will correspond to the image to form a joint feature for encoder input.

[0071] Use the VGG network to extract image features. Divide the feature map to generate a patch sequence, each patch corresponds to a pixel position in the image.

[0072] The VGG16 network is used as the image feature extractor. The output is a feature map after 5 layers of convolution, which captures the local features and texture information of the image.

[0073] The radar points are positionally encoded and projected to the corresponding position of the image feature map. The patch size is set as a function proportional to the radar detection distance to adapt to the characteristics of sparse points in millimeter-wave radar. The self-attention mechanism of the Transformer encoder captures the global relationship between radar and image features. The contextual information of image features is used to enhance the expressiveness of sparse radar points. Input sequence length: N = 100N = 100, containing m radar point encodings and Nm blank predictions with random completion.

[0074] Generate a region of interest based on radar points and project it onto the image feature map. Compress image features along the vertical dimension to reduce redundant information. Fuse the radar position sequence with image features and lock the target position through cross attention. The multi-head attention mechanism combines multi-modal features to enhance detection accuracy.

[0075] The decoder finally outputs N prediction results, each of which contains the target category c and the bounding box box.

[0076] The prediction set is matched one-to-one with the true target set using the Hungarian algorithm. The classification loss is the cross entropy loss, which is used to compare the predicted category with the true category. The bounding box loss is the L1 loss, which measures the error between the predicted bounding box and the true bounding box. The weighted sum of the classification loss and the bounding box loss is the total loss.

[0077] The optimization strategy is to use the AdamW optimizer, set the initial learning rate to 10^{-4}, and use cosine annealing learning rate scheduling to gradually reduce the learning rate.

[0078] This example is implemented based on PyTorch and runs on NVIDIA A100 GPU. The training set and test set are 80% and 20% respectively. The specific category data is shown in Table 1:

[0079] Table 1 Category data

[0080] category Baseline method (mAP) D-CRFNet(mAP) car 81.3％ 89.7％ pedestrian 74.2％ 82.4％ bike 69.8％ 78.5％ Traffic signs 77.6％ 85.3％

[0081] Across all target categories, D-CRFNet significantly improves detection accuracy and enhances robustness.

[0082] The D-CRFNet used in this invention shows stronger robustness in rainy days, nights and dense traffic scenes. The processing time per frame is 48ms, which meets the needs of real-time detection.

[0083] Through this example, it can be seen that the D-CRFNet network combined with the Transformer model and the fusion of multimodal features effectively improves the accuracy and robustness of target detection, and provides strong technical support for perception tasks in complex scenarios.

[0084] It should be noted that, in the present invention, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0085] The above embodiments are merely examples of the present invention and do not limit the protection scope of the present invention. All designs that are the same or similar to the present invention fall within the protection scope of the present invention.

Claims

1. A vehicle detection method based on millimeter wave radar and vision fusion, characterized in that: The following steps are involved: The acquired millimeter-wave radar data is filtered by setting filtering conditions to obtain valid radar data, and the data set is constructed by combining the acquired vehicle image data; Based on the D-CRFNet network, features are extracted from the data set to obtain data features, wherein the data features include image features and radar sequence features; A double-layer encoder is set to encode the data features, and the image features and radar sequence features are respectively input into different layers of the double-layer encoder, and after position encoding, they are respectively fused by weighted splicing to obtain fused features; Use the attention mechanism to process the fusion features and obtain enhanced fusion features; Set up a decoder to decode the enhanced fusion features to obtain the target boundary and column labels; By setting filtering conditions and non-maximum suppression, the obtained target boundaries and column labels are filtered and classified to obtain the vehicle target detection results.

2. The vehicle detection method based on millimeter wave radar and vision fusion as claimed in claim 1, characterized in that: The steps of processing the data features by the double-layer encoder include: The effective radar data is converted into a sequence, and the corresponding position code is generated according to the detection distance corresponding to the effective radar data to obtain the radar code data Position encoding is performed on the image features to obtain image encoding data, and the image encoding data is fused with the radar encoding data to obtain fusion features; The attention mechanism is a multi-head self-attention mechanism, which processes the fusion features formed by radar and image features to obtain enhanced fusion features; The enhanced fusion features are further processed using multi-layer perception to obtain the output of the encoder.

3. The vehicle detection method based on millimeter wave radar and vision fusion as claimed in claim 2, characterized in that: The obtaining of radar coded data comprises the following steps: Based on the valid radar data, determine the location information of each radar point, including x, y, z coordinates and timestamp; Project the three-dimensional position information of the radar point onto the image plane to obtain the corresponding position of each radar point in the image; Determine the corresponding patch size based on the distance from the radar point to the radar sensor; Get each patch on the image plane and arrange them into a sequence according to the order of each patch in the image; The features of each patch are extracted and combined with its position information in the image to generate a position coding sequence as radar coding data.

4. The vehicle detection method based on millimeter wave radar and vision fusion as claimed in claim 2, characterized in that: The step of obtaining the image coding data comprises the following steps: Obtaining the position of each pixel in the image feature; The position of each pixel is multiplied by two different sine functions with a period of 2Π, and two results are obtained as image coding data; one result is a sine function with a period of 2Π, and the other result is a cosine function with a period of 2Π.

5. The vehicle detection method based on millimeter wave radar and vision fusion as claimed in claim 1, characterized in that: The double-layer encoder includes a first-layer encoder and a second-layer encoder. The first-layer encoder uses the radar sequence as a query and the image feature sequence as a key and a value; the second-layer encoder uses the image feature sequence as a query and the radar sequence as a key and a value.

6. The vehicle detection method based on millimeter wave radar and vision fusion as claimed in claim 1, characterized in that: The decoding process includes: compressing the image features to obtain compressed image features; Project the position code of the radar feature sequence onto the image feature map to generate a region of interest; Use multi-head cross attention mechanism to extract features from the region of interest and lock the target position; A multi-head self-attention mechanism is used to accurately detect the target and obtain the target boundary and column label as the decoder output.

7. The vehicle detection method based on millimeter wave radar and vision fusion as claimed in claim 1, characterized in that: The number of the object boundaries is greater than the number of actual objects in the image.

8. A vehicle detection system based on millimeter wave radar and vision fusion, characterized in that: include: The data processing module is used to filter and screen the acquired millimeter-wave radar data by setting screening conditions to obtain effective radar data, and to construct a data set in combination with the acquired vehicle image data; Based on the D-CRFNet network, features are extracted from the data set to obtain data features, where the data features include image features and sequence features; The encoding module is used to set a double-layer encoder to encode the data features, input the image features and radar sequence features into different layers of the double-layer encoder respectively, perform position encoding respectively, and then perform weighted splicing and fusion to obtain fusion features; Use the attention mechanism to process the fusion features and obtain enhanced fusion features; The decoding module is used to set the decoder, decode the enhanced fusion features, and obtain the target boundary and column label; The detection result acquisition module is used to filter and classify the obtained target boundaries and column labels by setting filtering conditions and non-maximum suppression to obtain vehicle target detection results.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 7 is implemented.

Images