Tracking system and method based on millimeter wave radar end-to-end multi-target detection

Abstract

The invention discloses a tracking system and method based on millimeter wave radar end-to-end multi-target detection, and mainly solves the problems of low detection precision and poor tracking stability of pedestrians, non-motor vehicles and motor vehicles in a complex traffic scene in the prior art. The system comprises a radar signal processing unit, a radar detection and identification network and a DeepSORT tracking module. The radar detection and identification network adopts a hybrid architecture comprising an encoder and a decoder, and the encoder comprises a 3D CNN network and an expansion convolution DAM module; the decoder comprises a T-W / SW-MSA module and a full connection layer, according to the DeepSORT tracking module, a radar cross section RCS value and micro-Doppler radar physical characteristics are introduced, cascade matching and confirmation of a new trajectory state are added, so that the stability and robustness of target tracking are improved, and long-term stable tracking of a target is achieved by fusing a target detection result and combining a Hungary algorithm and a Kalman filtering technology. The method is high in detection precision and good in tracking stability, and can be used for intelligent traffic and automatic driving.

Description

Technical Field

[0001] This invention belongs to the field of radar technology, specifically relating to a multi-target recognition and tracking system that can be used in intelligent transportation and autonomous driving. Background Technology

[0002] With the rapid development of intelligent transportation and autonomous driving technologies, environmental perception systems, as a core component, directly impact the safety and reliability of the system. Millimeter-wave radar, due to its all-weather operation unaffected by lighting, rain, fog, or other environmental factors, its high-precision ranging and speed measurement capabilities, and its low cost, has become a crucial sensor in intelligent transportation and autonomous driving. However, millimeter-wave radar point cloud data is characterized by sparsity and noise interference. Traditional target detection and tracking methods, such as Kalman filtering and DBSCAN clustering, suffer from performance limitations in complex scenarios involving target occlusion and intersections, making it difficult to meet the demands for high precision and robustness.

[0003] In recent years, deep learning technology has made significant progress in the field of computer vision, especially the successful application of Convolutional Neural Networks (CNNs) and Transformers in target detection and tracking tasks, providing new ideas for millimeter-wave radar data processing. Combining deep learning with millimeter-wave radar can significantly improve the accuracy of target detection and the stability of tracking. For example, 3D CNN-based methods can effectively extract the spatiotemporal features of radar point clouds, while Transformers further enhance global feature extraction capabilities by modeling long-range dependencies through a self-attention mechanism. However, existing technologies still suffer from high computational complexity and insufficient anti-interference capabilities, necessitating an efficient and robust solution.

[0004] Patent application number 202110123456.7 discloses a millimeter-wave radar target detection and tracking method based on 3D CNN and LSTM. This method improves detection accuracy through multi-frame point cloud data fusion and utilizes LSTM to model the target's motion trajectory. However, this method has high computational complexity and limited tracking capability for non-linearly moving targets.

[0005] Patent application number 202210654321.0 discloses a Transformer-based millimeter-wave radar target detection method, which utilizes the Transformer's self-attention mechanism to extract global features from radar point clouds, overcoming the shortcomings of traditional CNN methods in global modeling capabilities. However, this method's performance is limited in practical applications because it does not fully consider the sparsity and noise interference of radar point clouds. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of the existing technology by proposing a tracking system and method based on end-to-end multi-target detection using millimeter-wave radar, so as to improve the detection accuracy and tracking stability of pedestrians, non-motorized vehicles and motor vehicles in complex traffic scenarios.

[0007] The key technology of this invention is: establishing a tracking experimental system based on millimeter-wave radar end-to-end multi-target detection through a heterogeneous fusion detection network and an improved DeepSORT algorithm, thereby achieving the above-mentioned objective of this invention. The implementation scheme includes:

[0008] 1. A tracking system based on end-to-end multi-target detection using millimeter-wave radar, comprising a radar signal processing unit, a radar detection and recognition network, and a DeepSORT tracking module, characterized in that:

[0009] The radar detection and recognition network adopts a hybrid architecture including an encoder and a decoder. The encoder includes a 3DCNN network and a DAM module with dilated convolutions, which are used to extract local spatiotemporal features and realize multi-scale feature fusion, respectively. The decoder includes a TW / SW-MSA module and a fully connected layer, which are used to dynamically fuse its own global features and the local features of the encoder.

[0010] The DeepSORT tracking module improves upon the original SORT algorithm by introducing radar cross section (RCS) values ​​and micro-Doppler radar physical characteristics, and adding cascaded matching and confirmation of new trajectory states to enhance the stability and robustness of target tracking. Furthermore, by fusing the target detection results and combining the Hungarian algorithm with Kalman filtering technology, it achieves long-term stable tracking of the target.

[0011] Furthermore, the 3D CNN network includes 3D convolutional layers, 3D pooling layers, fully connected layers, and activation functions.

[0012] The algorithm utilizes 3D convolution kernels to simultaneously extract features from the length, width, and depth dimensions of radar spectra, capturing correlation information in three-dimensional space.

[0013] Furthermore, the dilated convolutional DAM module includes a dilation rate of 2 and an equivalent kernel size of 5×5.

[0014] Convolution, by expanding the receptive field, fuses the range, velocity, and temporal multi-scale features of radar.

[0015] Furthermore, the TW / SW-MSA module includes a multi-head self-attention mechanism based on windows and sliding windows. This mechanism is used to perform multi-head self-attention operations within each window after the feature map is divided into multiple non-overlapping small windows, and to slide and cyclically shift the windows to perform multi-head self-attention operations between sliding windows, thereby realizing cross-window information flow.

[0016] 2. A tracking method based on end-to-end multi-target detection using millimeter-wave radar, characterized in that it includes:

[0017] (1) Preprocessing of the original signal:

[0018] The raw 77GHz FMCW radar signal is preprocessed, including dynamic noise suppression and clutter removal through CFAR threshold filtering, to generate multi-frame radio frequency images with dimensions of 16×128×128×2, providing spatiotemporal sequence input for the detection network;

[0019] (2) Target detection:

[0020] Construct a detection network that includes an encoder and a decoder;

[0021] The local spatiotemporal features of pedestrian gait and vehicle acceleration are extracted by the 3D CNN network in the encoder. The DAM module in the encoder is used to perform dilated convolution to expand the receptive field and fuse radar distance, velocity and time scale features. Sliding window attention is used to reduce cross-frame velocity dependence.

[0022] The encoder's output key-value pairs and the decoder's query are dynamically fused through the TW / SW-MSA module in the decoder, and the target localization is optimized by combining dynamic weight parameters. The target is then classified using a fully connected layer, and the target category is output.

[0023] (3) Target tracking:

[0024] The detection results are fed into the improved DeepSORT module for prediction, matching, association, and updating:

[0025] Kalman filtering is used to predict the target tracking box position in the next frame of the radar detection image;

[0026] By introducing radar cross section (RCS) values ​​and micro-Doppler radar physical characteristics, and adding cascaded matching and trajectory confirmation states, the target detection box and prediction box are correlated. The optimal pairing of prediction and detection boxes is then achieved using the Hungarian algorithm.

[0027] If a match is successful, the predicted bounding box is directly used as the tracking box for the current target.

[0028] If a match cannot be found, the tracking box is updated at the current target tracking position using Kalman filtering to improve the detection accuracy and tracking stability of pedestrians, non-motorized vehicles, and motor vehicles in complex traffic scenarios.

[0029] Compared with the prior art, the present invention has the following advantages:

[0030] First, the system of the present invention, by adopting a network structure of encoder and decoder, can simultaneously capture the local and global features of the target, resulting in a wider detection range and stronger anti-interference ability. By employing a sliding window attention mechanism, W-MSA / SW-MSA modules and dilated convolutional DAM modules, the computational complexity can be significantly reduced and the computational efficiency improved.

[0031] Secondly, the system of the present invention adopts an improved DeepSORT tracking module, which introduces radar cross section (RCS) value and micro-Doppler radar physical characteristics, and increases cascade matching and confirmation of new trajectory states, thereby improving the stability of tracking pedestrians, non-motorized vehicles and motor vehicles in complex traffic scenarios.

[0032] Simulation tests show that the system of the present invention has higher detection accuracy and better tracking stability in complex scenarios such as intersections, highways, and rainy / foggy weather. It solves the performance bottleneck of traditional cameras and lidar in low visibility environments, and the present invention has a wider range of applicable scenarios. Attached Figure Description

[0033] Figure 1 This is a block diagram of the tracking system structure based on end-to-end multi-target detection using millimeter-wave radar according to the present invention;

[0034] Figure 2 This is a schematic diagram of the radar target detection and recognition network in the system of the present invention;

[0035] Figure 3 This is a schematic diagram of the working principle of the improved DeepSORT module in the system of this invention;

[0036] Figure 4 This is a flowchart illustrating the implementation of the tracking method based on end-to-end multi-target detection using millimeter-wave radar according to the present invention. Detailed Implementation

[0037] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention and not all embodiments. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative effort should all fall within the protection scope of the present invention.

[0038] Example 1: Tracking system based on end-to-end multi-target detection of millimeter-wave radar.

[0039] Reference Figure 1 This example includes a radar signal processing unit, a radar detection and identification network, and an improved DeepSORT tracking module.

[0040] The radar signal processing unit is used to preprocess the original radar signal to obtain a radar spectrum including range-Doppler RD, range-azimuth RA, and range-angle-Doppler RAD spectra. Its specific implementation includes:

[0041] 1.1) To remove noise and clutter from radar signals and retain valid target signals, a constant false alarm rate (CFAR) algorithm is used to dynamically adjust the detection threshold to adapt to changes in background noise.

[0042] First, a sliding window is set for the radar signal, and the background noise level in each window is calculated.

[0043] Subsequently, based on the background noise level within the window and the pre-set false alarm rate, the detection threshold is dynamically adjusted, and signal points with amplitudes exceeding the threshold are retained while noise points with amplitudes below the threshold are suppressed, so as to effectively remove clutter while retaining the target signal.

[0044] 1.2) In order to normalize the amplitude of the radar signal to a fixed range of [0, 1], avoid the impact of uneven data distribution on model training, and improve the stability and convergence speed of model training, the original data is normalized and standardized to obtain the linearly normalized amplitude. and the standardized amplitude :

[0045] ,

[0046] ,

[0047] in, Indicates the amplitude of the radar signal. Indicates the minimum amplitude. Indicates the maximum amplitude. This represents the mean. Indicates standard deviation;

[0048] 1.3) Perform Fast Fourier Transform (FFT) on the range dimension of the radar signal to extract range information, perform FFT on the velocity dimension to extract Doppler information, and perform FFT on the angle dimension to extract angle information, thereby obtaining the range-angle-Doppler (RAD) radar spectrum, which serves as the input to the radar detection and identification network.

[0049] The radar detection and identification network, such as Figure 2As shown, it employs a hybrid architecture including an encoder and a decoder. The encoder comprises a W / SW-MSA, downsampling, a 3D CNN network, and a DAM module with dilated convolutions, used to extract local spatiotemporal features and achieve multi-scale feature fusion. Specifically, W / SW-MSA divides the input radar spectrogram into multiple local windows, extracting features within each window; downsampling reduces the feature size while increasing the number of feature channels, expanding the receptive field and allowing subsequent feature processing modules to capture more global information; the 3D CNN network includes 3D convolutional layers, 3D pooling layers, fully connected layers, and activation functions, utilizing 3D convolutional kernels to simultaneously extract features from the length, width, and depth dimensions of the radar spectrogram, extracting local spatiotemporal features such as pedestrian gait and vehicle acceleration, and capturing correlation information in three-dimensional space; the DAM module with dilated convolutions includes dilated convolutions with a dilation rate of 2 and an equivalent kernel of 5×5, which expands the receptive field to fuse radar range, velocity, and temporal multi-scale features.

[0050] The decoder includes upsampling, a TW / SW-MSA module, and a fully connected layer for dynamically fusing its own global features with the encoder's local features; among which:

[0051] The TW / SW-MSA module includes a multi-head self-attention mechanism based on windows and sliding windows. This mechanism performs multi-head self-attention operations within each window after the feature map is divided into multiple non-overlapping small windows. It also performs sliding and cyclic shifting of the windows to execute multi-head self-attention operations between sliding windows, enabling information flow across windows. Its formula is expressed as follows:

[0052] ,

[0053] in, Indicates based on query ,key and value The results of multi-head self-attention calculation; This represents the concatenation of all attention head results. express

[0054] The output of the i-th attention head, where i represents 1 to h, and h is the number of attention heads; Represents the projection matrix;

[0055] The fully connected layer dynamically fuses the key-value pairs output by the encoder and the query by the decoder through a cross-attention mechanism, and optimizes the target localization by combining dynamic weight parameters. Finally, it outputs the bounding box and class probability, which are expressed by the following formula:

[0056] ,

[0057] in, This represents the result calculated using an attention mechanism based on query Q, key K, and value V.

[0058] T stands for transpose. Indicates the scaling factor. Indicates the activation function;

[0059] The improved DeepSORT tracking module is an improvement on the original SORT algorithm, used for prediction, matching, and updating of detection results. It includes Kalman filter (EKF) prediction, data association, Hungarian algorithm optimal pairing, and trajectory management, such as... Figure 3 As shown.

[0060] The extended Kalman filter (EKF) is used to predict the nonlinear motion information of the target to obtain the prediction box.

[0061] This data association process introduces radar cross section (RCS) values ​​and micro-Doppler radar physical characteristics, fuses multi-dimensional features to construct a cost matrix, and adds cascaded matching to associate the target detection box with the prediction box.

[0062] The Hungarian algorithm achieves optimal pairing of predicted and detected bounding boxes by inputting a pre-constructed cost matrix into the algorithm and solving for the optimal bipartite matching of the cost matrix. Its implementation includes:

[0063] The cost matrix is ​​mapped to a bipartite graph structure, and the augmenting path is iteratively searched in the bipartite graph to gradually filter the edge combinations that satisfy "each node matches only once";

[0064] During the iteration process, the total cost of the current matching combination is continuously calculated, and finally the edge combination with the minimum total cost is selected. This combination is the optimal bisection matching result of the cost matrix.

[0065] The optimal binary matching results are used to form the best pairing index between tracking boxes and detection boxes. Each pair of indexes clearly corresponds to the optimal matching relationship between a tracking box and a detection box, and at the same time, the unmatched tracking boxes and detection boxes are returned.

[0066] This trajectory management process incorporates a trajectory status confirmation mechanism, which categorizes trajectories into confirmed and unconfirmed states based on the number of matches. Newly generated trajectories are in the unconfirmed state, and unconfirmed trajectories that match the detection results more than three times consecutively can be converted to the confirmed state. This reduces trajectories generated by false detections, enables dynamic updates of the tracking frame and trajectory management, and ultimately outputs stable traffic participant category tracking IDs and real-time location information. This keeps the ID switching rate below 7 times per minute, improving the tracking stability of pedestrians, non-motorized vehicles, and motor vehicles in complex traffic scenarios.

[0067] Example 2: Tracking method based on end-to-end multi-target detection of millimeter-wave radar

[0068] Reference Figure 4 The implementation of this example includes the following steps:

[0069] Step 1: Collect raw data from the millimeter-wave radar.

[0070] The raw signals of the millimeter-wave radar with an operating frequency of 77 GHz and a transmitted waveform of FMCW were acquired and preprocessed, including dynamic noise suppression by removing clutter through CFAR threshold filtering.

[0071] The amplitude of the suppressed signal is linearly normalized and standardized to fix it in the range of [0, 1], so as to avoid the impact of uneven data distribution on model training and improve the stability and convergence speed of model training.

[0072] The normalized and standardized signals are subjected to multi-dimensional fast Fourier transform to generate multi-frame range-angle-Doppler radar spectra with dimensions of 16×128×128×2.

[0073] Step 2: Target detection and identification are performed on the preprocessed radar spectrogram.

[0074] The preprocessed radar spectrogram is fed into a radar target detection and recognition network that includes an encoder and a decoder. The local spatiotemporal features of pedestrian gait and vehicle acceleration are extracted by the 3D CNN network in the encoder. The DAM module in the encoder is used to perform dilated convolution to expand the receptive field and fuse radar distance, velocity and time scale features. Sliding window attention is used to reduce cross-frame velocity dependence.

[0075] The encoder's output key-value pairs and the decoder's query are dynamically fused by the TW / SW-MSA module in the decoder, and the target localization is optimized by combining dynamic weight parameters. The target is then classified using a fully connected layer, and the target category is output.

[0076] Step 3: Input the detection results into the improved DeepSORT module for prediction, cost matrix construction, trajectory management, data matching and correlation, and updates to achieve target tracking.

[0077] 3.1) The position of the target tracking box in the next frame of the radar detection image is predicted using a Kalman filter to form a predicted trajectory;

[0078] 3.2) Introducing radar cross section (RCS) values ​​and micro-Doppler radar physical characteristics, the RCS, micro-Doppler features, appearance features, and motion features are fused to obtain multi-dimensional feature vectors for the detection box and the prediction box, and their difference is calculated, i.e., the feature cost matrix. ;

[0079] The multidimensional motion vector, i.e., the motion cost matrix, is calculated based on the position and velocity differences predicted by the Kalman filter. Set the threshold for the feature cost matrix and motion cost matrix threshold The feature cost matrix and motion are respectively

[0080] The cost matrix is ​​compared with its respective threshold to determine whether the detection box and the predicted box are related:

[0081] like and Then, the detection box and the prediction box are associated with each other, and the cost matrix is ​​obtained by weighted summation of the feature cost matrix and the motion cost matrix. The association matrix is ​​the cost matrix.

[0082] like and If this is not the case, the detection box and the prediction box cannot be associated with each other;

[0083] 3.3) Add cascading matching and trajectory confirmation status. The trajectory is divided into recently active status and potentially occluded status according to the update status. During the data matching process, the recently active status is matched first to reduce the probability of mismatch under long-term occlusion.

[0084] 3.4) Based on the number of matches, the trajectories are divided into confirmed and unconfirmed states. Newly generated trajectories are classified as unconfirmed states, and unconfirmed trajectories that are matched with the detection results more than three times consecutively are classified as confirmed states.

[0085] 3.5) Associate the target's detection bounding box with the predicted bounding box to determine whether the current trajectory is a confirmed trajectory:

[0086] If it is a confirmed trajectory, the detection box is used to correct the prediction box first, and then the position, velocity and other information of the trajectory are updated through Kalman filtering;

[0087] If the trajectory is not in a confirmed state, the number of matches for the current trajectory is accumulated. When the number of consecutive matches is greater than or equal to 3, the trajectory is changed from "temporary state" to "confirmed state", and then the trajectory is updated by Kalman filtering.

[0088] 3.6) Input the constructed cost matrix into the Hungarian algorithm to solve for the optimal bipartite matching of the cost matrix, thereby achieving the best pairing of the predicted bounding box and the detection bounding box:

[0089] 3.6.1) Map the cost matrix to a bipartite graph structure, and gradually filter edge combinations that satisfy "each node matches only once" by iteratively searching for augmenting paths in the bipartite graph;

[0090] 3.6.2) During the iteration process, the total cost of the current matching combination is continuously calculated, and finally the edge combination with the minimum total cost is selected. This combination is the optimal bisection matching result of the cost matrix.

[0091] 3.6.3) Utilize the optimal binary matching results to form the best pairing index between tracking boxes and detection boxes. That is, each pair of indexes has clearly defined the optimal matching relationship between a tracking box and its corresponding detection box, and at the same time returns the unmatched tracking boxes and detection boxes.

[0092] 3.7) Determine whether the predicted bounding box and the detection bounding box are successfully matched using the Hungarian algorithm:

[0093] If a matching index is generated through optimal binary matching, i.e. a successful match is achieved, the predicted bounding box is directly used as the tracking box of the current target.

[0094] If a matching index cannot be generated, i.e., no match can be found, the tracking box is updated at the current target tracking position using Kalman filtering to improve the detection accuracy and tracking stability of pedestrians, non-motorized vehicles and motor vehicles in complex traffic scenarios.

[0095] 3.8) Set the maximum number of lost frames and check if the number of trajectory frames exceeds the maximum number of lost frames:

[0096] If the number of trajectory frames exceeds the maximum number of lost frames, then delete the trajectory;

[0097] Otherwise, the trajectory is preserved to obtain the tracking trajectories of pedestrians, non-motorized vehicles, and motor vehicles.

[0098] It should be noted that the flowchart representations or method representations of the above embodiments can be understood as representing a module, segment, or portion of code comprising one or more executable instructions configured to implement a specific logical function or process. This invention is not limited to the disclosed preferred embodiments, and its implementation may not follow the order shown or discussed. That is, the step numbers in the specification and claims are only for clear description and understanding of the embodiments of this invention, and their order is not limited.

[0099] The effects of this invention can be further illustrated by the following simulation results:

[0100] I. Simulation Conditions:

[0101] Dataset: Training set percentage: 92%, Test set percentage: 8%;

[0102] Hardware environment: The millimeter-wave radar model is IWR1843, and the millimeter-wave radar operates at a frequency of 77 GHz;

[0103] The transmitted signal is a linear frequency modulated continuous wave (FMCW).

[0104] Software environment: GPU: NVIDIA RTX 3080Ti, Network framework: PyTorch;

[0105] Training parameters: Optimizer: Adam;

[0106] Initial learning rate: 0.0001;

[0107] Learning rate decay strategy: cosine annealing;

[0108] Normalization method: Group Normalization;

[0109] Loss function: Smooth-L1 Loss;

[0110] II. Simulation Content and Results

[0111] Simulation 1: Under the above conditions, the present invention was used to collect raw millimeter-wave radar data on campus roads, identify traffic participants through a detection network, and perform target tracking using an improved DeepSORT module. The performance indicators are shown in Table 1.

[0112] Table 1 Performance indicators of the present invention for target tracking on campus roads

[0113]

[0114] Simulation 2: Under the above conditions, the present invention was used to collect raw millimeter-wave radar data on urban streets, identify traffic participants through a detection network, and perform target tracking using an improved DeepSORT module. The performance indicators are shown in Table 2.

[0115] Table 2 Performance indicators of the present invention for target tracking in urban streets

[0116]

[0117] As can be seen from Tables 1 and 2, the present invention has higher detection accuracy, better tracking stability, and faster processing speed in campus road scenarios with sparse targets and regular movement. However, in urban road scenarios with dense targets, severe occlusion, and complex movement, the tracking accuracy and stability decrease, and the number of missed detections and false detections increases, but the processing speed can still meet the real-time requirements.

Claims

1. A tracking system based on end-to-end multi-target detection using millimeter-wave radar, comprising a radar signal processing unit, a radar detection and recognition network, and a DeepSORT tracking module, characterized in that: The radar detection and recognition network adopts a hybrid architecture including an encoder and a decoder. The encoder includes a 3D CNN network and a DAM module with dilated convolutions, which are used to extract local spatiotemporal features and realize multi-scale feature fusion, respectively. The decoder includes a TW / SW-MSA module and a fully connected layer, which are used to dynamically fuse its own global features and the local features of the encoder. The DeepSORT tracking module improves upon the original SORT algorithm by introducing radar cross section (RCS) values ​​and micro-Doppler radar physical characteristics, and adding cascaded matching and confirmation of new trajectory states to enhance the stability and robustness of target tracking. Furthermore, by fusing the target detection results and combining the Hungarian algorithm with Kalman filtering technology, it achieves long-term stable tracking of the target.

2. The system according to claim 1, characterized in that, The 3D CNN network includes 3D convolutional layers, 3D pooling layers, fully connected layers, and activation functions. It uses 3D convolutional kernels to extract features from the length, width, and depth of the radar spectrum simultaneously, capturing correlation information in three-dimensional space.

3. The system according to claim 1, characterized in that, The DAM module of the dilated convolution includes a dilated convolution with a dilation rate of 2 and an equivalent kernel of 5×5, which expands the receptive field to fuse the range, velocity and time multi-scale features of the radar.

4. The system according to claim 1, characterized in that, The TW / SW-MSA module includes a multi-head self-attention mechanism based on windows and sliding windows. This mechanism performs multi-head self-attention operations within each window after the feature map is divided into multiple non-overlapping small windows. It also performs sliding and cyclic shifting of the windows to execute multi-head self-attention operations between sliding windows, enabling information flow across windows. The formula is as follows: ， in, Indicates based on query ,key and value The results of multi-head self-attention calculation; This indicates the concatenation of all attention head results. This represents the output of the i-th attention head, where i represents numbers from 1 to h, and h is the number of attention heads. This represents the projection matrix.

5. The system according to claim 1, characterized in that, The decoder dynamically fuses its global features with the encoder's local features through a cross-attention mechanism. This fusion combines the key-value pairs output by the encoder with the decoder's query, and optimizes target localization using dynamic weight parameters. The final output is a bounding box and class probability, expressed by the following formula: ， in, This represents the result calculated using an attention mechanism based on query Q, key K, and value V. T stands for transpose. Indicates the scaling factor. This represents the activation function.

6. The system according to claim 1, characterized in that, The implementation of adding cascaded matching and confirming the new trajectory state includes: Trajectories are categorized into recently active and potentially occluded states based on their update status, and these are prioritized for matching during the data matching process. Recent activity status is used to reduce the probability of false matches under long-term occlusion. Trajectories are categorized into confirmed and unconfirmed states based on the number of matches. Newly generated trajectories are in the unconfirmed state. Trajectories in the unconfirmed state that match the detection results more than three times consecutively can be converted into confirmed states. This reduces the number of false detections and ensures the accuracy and stability of the tracking results.

7. The system according to claim 1, characterized in that, The DeepSORT tracking module achieves long-term stable tracking of the target by fusing the target detection results and combining the Hungarian algorithm with Kalman filtering technology. Its implementation includes: The detection results are input into the improved DeepSORT module, in which the extended Kalman filter (EKF) is adjusted according to the detection... The measurement results are used to predict the nonlinear motion trajectory of the target and obtain its prediction box. Data association is achieved by using the Hungarian algorithm to optimally pair predicted bounding boxes with detection boxes, thereby classifying traffic participants. The ID switching rate was reduced to 7 times / minute, maintaining long-term stability in target tracking.

8. A tracking method based on end-to-end multi-target detection using millimeter-wave radar, characterized in that, include: (1) Preprocessing of the original signal: The raw 77GHz FMCW radar signal is preprocessed, including dynamic noise suppression and clutter removal through CFAR threshold filtering, to generate multi-frame radio frequency images with dimensions of 16×128×128×2, providing spatiotemporal sequence input for the detection network; (2) Target detection: Construct a detection network that includes an encoder and a decoder; The local spatiotemporal features of pedestrian gait and vehicle acceleration are extracted by the 3D CNN network in the encoder. The DAM module in the encoder is used to perform dilated convolution to expand the receptive field and fuse radar distance, velocity and time scale features. Sliding window attention is used to reduce cross-frame velocity dependence. The encoder's output key-value pairs and the decoder's query are dynamically fused through the TW / SW-MSA module in the decoder, and the target localization is optimized by combining dynamic weight parameters. The target is then classified using a fully connected layer, and the target category is output. (3) Target tracking: The detection results are fed into the improved DeepSORT module for prediction, matching, association, and updating: The position of the target tracking box in the next frame of the radar detection image is predicted using a Kalman filter; By introducing radar cross section (RCS) values ​​and micro-Doppler radar physical characteristics, and adding cascaded matching and trajectory confirmation states, the target detection box and prediction box are correlated. The optimal pairing of prediction and detection boxes is then achieved using the Hungarian algorithm. If a match is successful, the predicted bounding box is directly used as the tracking box for the current target. If a match cannot be found, the tracking box is updated at the current target tracking position using Kalman filtering to improve the detection accuracy and tracking stability of pedestrians, non-motorized vehicles, and motor vehicles in complex traffic scenarios.

9. The method according to claim 8, characterized in that, The process of associating the detection box and prediction box of the target in (3) includes the following: 3a) Fuse RCS, micro-Doppler features, appearance features, and motion features to obtain multi-dimensional feature vectors for the detection box and the predicted box, and calculate their difference, i.e., the feature cost matrix. At the same time, a threshold for the feature cost matrix is ​​set. ; 3b) Calculate the multidimensional motion vector, i.e., the motion cost matrix, based on the position and velocity differences predicted by the Kalman filter. At the same time, a threshold for the motion cost matrix is ​​set. ; 3c) Compare the feature cost matrix and motion cost matrix with their respective thresholds to determine whether the detection box and the predicted box are related: like and Then, the detection box and the prediction box are associated with each other, and the cost matrix is ​​obtained by weighted summation of the feature cost matrix and the motion cost matrix. The association matrix is ​​the cost matrix. like and If this happens, the detection box and the prediction box cannot be associated with each other.

10. The method according to claim 8, characterized in that, In step (3), the optimal pairing of the tracking box and the detection box using the Hungarian algorithm involves inputting the constructed cost matrix into the Hungarian algorithm to solve for the optimal bipartite matching of the cost matrix. The cost matrix is ​​mapped to a bipartite graph structure, and the augmenting path is iteratively searched in the bipartite graph to gradually filter the edge combinations that satisfy "each node is matched only once"; During the iteration process, the total cost of the current matching combination is continuously calculated, and finally the edge combination with the minimum total cost is selected. This combination is the optimal bisection matching result of the cost matrix. The optimal binary matching results are used to form the best pairing index between tracking boxes and detection boxes. Each pair of indexes clearly corresponds to the optimal matching relationship between a tracking box and a detection box, and unmatched tracking boxes and detection boxes are returned.

Images