Automatic driving target detection and tracking method based on multi-source heterogeneous information fusion

Abstract

The application discloses an automatic driving target detection and tracking method based on multi-source heterogeneous information fusion, which comprises the following steps: S1, time-space registration of data collected by a vehicle-mounted millimeter wave radar and a camera, and projection of millimeter wave radar points into a camera image coordinate system; S2, construction of a radar feature map to provide a spatial attention module for a camera target detection algorithm; S3, introduction of a SENET channel attention module and a radar feature map spatial attention module into a backbone structure of a target detection algorithm YOLOv5; S4, construction of a KCF target tracking algorithm that fuses an extended Kalman filter and a data association algorithm; and S5, under the condition that the camera is not blocked, fusion of a target detection algorithm and a target tracking algorithm for vehicle target tracking, and under the condition that the camera is blocked, vehicle target tracking based on the target tracking algorithm. The automatic driving target detection and tracking method can realize accurate detection and stable tracking of a vehicle target.

Description

Technical Field

[0001] This invention relates to the field of autonomous driving technology, and in particular to an autonomous driving target detection and tracking method based on multi-source heterogeneous information fusion. Background Technology

[0002] In recent years, traffic accidents and congestion have occurred frequently, posing a significant threat to human life and property. With continuous technological advancements, autonomous driving has become a key means of solving these problems. Target detection and tracking technologies play a crucial role in the field of autonomous driving; however, due to the complexity and dynamic nature of the driving environment and the limited capabilities of a single sensor, it is necessary to integrate multiple sensors to ensure robust, accurate, and reliable target detection and tracking.

[0003] In the field of autonomous driving object detection, thanks to the rapid development of visual object detection technology, multi-sensor fusion detection technology has gradually shifted from a radar-based, camera-assisted strategy to a radar-assisted, camera-based strategy. Leveraging radar's strong target localization capabilities to assist camera-based object detection algorithms has become a research focus in recent years. The YOLOv5 object detection algorithm, a real-time and efficient single-stage object detection algorithm developed in the last two years, is well-suited for the dynamic road environments of autonomous driving. If the detection network is informed in advance which regions of the image to prioritize detection, the detection speed and accuracy can be further improved.

[0004] In the field of autonomous driving target tracking, traditional vision-based tracking technologies are not very stable, especially when the target is obscured by obstacles, which can easily lead to camera tracking failures. Therefore, using multi-sensor fusion tracking methods is an effective measure to improve the stability of autonomous driving target tracking. Utilizing multiple sensors such as radar and cameras to track targets can meet the requirements for continuous and stable target tracking under different road conditions. Summary of the Invention

[0005] Based on the above background, this invention processes camera data using the YOLOv5 target detection algorithm and radar data using the KCF target tracking algorithm. It also employs a strategy of using radar as a secondary and camera as the primary sensor to fuse the data from both sensors, thereby achieving accurate detection and stable tracking of vehicle targets.

[0006] The autonomous driving target detection and tracking method based on multi-source heterogeneous information fusion includes the following steps:

[0007] S1. Perform time and space registration on the data collected by the vehicle-mounted millimeter-wave radar and the camera, and then project the millimeter-wave radar points into the camera image coordinate system through coordinate transformation.

[0008] S2. Construct radar feature maps and leverage the strong positioning capabilities of radar to provide a spatial attention module for camera target detection algorithms.

[0009] S3. The SENET channel attention module and the radar feature map spatial attention module are introduced into the backbone structure of the target detection algorithm YOLOv5, so that the target detection algorithm pays more attention to the area where the target may exist, thereby improving the speed and accuracy of target detection.

[0010] S4. Construct a KCF target tracking algorithm that integrates extended Kalman filtering and data association algorithms;

[0011] S5. When the camera is not obstructed, the target detection algorithm and the target tracking algorithm are integrated to track the vehicle target; when the camera is obstructed, the target tracking algorithm is used to track the vehicle target.

[0012] Furthermore, the specific processing procedure for step S1 is as follows:

[0013] S11. To achieve time registration between the millimeter-wave radar and the camera, since the sampling frequencies of the two sensors are different, the time registration problem between the two sensors is solved by triggering the camera to sample after the radar samples.

[0014] S12. To achieve spatial registration between the millimeter-wave radar and the camera, since the two sensors describe the target's coordinates differently, it is necessary to unify the data from both sensors into the same coordinate system. The spatial transformation model between the vehicle-mounted millimeter-wave radar and the camera can be represented as follows:

[0015]

[0016] In the formula, [x r y r z r 1] T [uv 1] represents the target's coordinates in millimeter-wave radar coordinates. T Let R be the target coordinates in pixel coordinates; R is a 3×3 identity orthogonal rotation matrix; T is a 3×1 translation matrix; I is the identity matrix; d x With d y These represent the size of each pixel in physical units on the horizontal and vertical axes, respectively; f is the camera's focal length; k is the scaling factor; x r y r z r Let u and v be the three-dimensional coordinates of the target in the millimeter-wave radar coordinate system; let z be the two-dimensional coordinates of the target in the pixel coordinate system. c The coordinates are in the camera coordinate system; u0 and v0 are the two-dimensional coordinates in the pixel coordinate system before the coordinate system rotation transformation operation.

[0017] S13. Based on the registered data, project the millimeter-wave radar points onto the camera image.

[0018] Furthermore, the specific processing procedure for step S2 is as follows:

[0019] S21. Generate a blank single-channel radar feature map of the same size as the image, and set the initial value of each pixel in the radar feature map to 0.

[0020] S22. After initializing the value of each pixel in the radar feature map to 0, construct the Region of Interest (ROI) centered on a single radar projection point in the radar feature map, which is the possible location of the target detected by the radar. Then set the pixel value in the ROI of the radar feature map to 1.

[0021] S23. When the target distance is closer or the target size is larger, there are more radar reflection points after clustering. This can be seen as the area of ​​the ROI region being linearly related to the number of radar points after clustering.

[0022] S=αn+β

[0023] In the formula, n is the number of radar reflection points; α and β are hyperparameters;

[0024] α and β are learned by maximizing the IOU between the ROI and the ground truth bounding box, as shown in the following formula:

[0025]

[0026] In the formula, N is the number of input samples during the hyperparameter learning process; M i The number of bounding boxes for the object in the i-th sample;

[0027] The formula for calculating IOU is as follows:

[0028]

[0029] In the formula, ROI represents the region of interest; GT represents the ground truth bounding box; and S represents the area to be calculated.

[0030] S24. Finally, a radar feature map representing the target location information is generated, with pixel values ​​of 1 in the region of interest of the ROI and pixel values ​​of 0 in other parts.

[0031] Furthermore, the specific processing procedure for step S3 is as follows:

[0032] S31, SENET stands for Squeeze-and-Excitation Networks. The Squeeze part compresses the input (H,W,C) feature map into (1,1,C), and the Excitation part feeds the compressed (1,1,C) feature map into a fully connected layer to predict the importance of each channel. After determining the importance of different channels, the values ​​are applied to the corresponding channels of the input feature map. This allows the object detection network to apply different attention to different channels of the feature map. In other words, SENET provides different weights to different channels of the input feature map, making the object detection algorithm more efficient and accurate.

[0033] S32. Add the SENET channel attention module to the last layer of the backbone structure in the YOLOv5 object detection algorithm. Multiply the channel weight values ​​with the two-dimensional matrix of the corresponding channel in the feature map, so that the network can focus more on the channel with large weight in the feature map.

[0034] S33. Multiply the feature map processed by the SENET channel attention module with the radar feature map so that the feature map is simultaneously affected by the channel attention weight and the spatial location attention weight, thereby constructing a channel-space attention mechanism module that fuses radar information.

[0035] S34. Insert the channel-spatial attention module into the last layer of the backbone structure of the YOLOv5 object detection algorithm to obtain the candidate regions and prior information where the target may exist, so that the object detection algorithm can detect the target more quickly and efficiently.

[0036] Furthermore, the specific processing procedure for step S4 is as follows:

[0037] S41. In the radar detection coordinate system, the vehicle target is modeled using the Bicycle dynamics model. The equations of motion describing the vehicle's motion state, and the relative motion between the main vehicle and the target vehicle in front, are as follows:

[0038]

[0039] In the formula, a and b are the input quantities of the model; B is the wheelbase of the preceding vehicle; and v is the relative speed between the main vehicle and the preceding vehicle. θ is the front wheel steering angle of the vehicle in front; θ is the direction angle of the vehicle in front; x r y r The vehicle's current coordinates;

[0040] Discretizing the above equation, we get:

[0041]

[0042] In the formula, δ a With δ b For interference noise; δ P With δ s The multiplicative noise represents the increase in uncertainty of the vehicle's motion state as the vehicle's steering angle and speed change; the above four types of noise are independent of each other and are all Gaussian white noise with a mean of 0 and a known covariance.

[0043] The measurement equation for the millimeter-wave radar at time k is:

[0044]

[0045] In the formula, the measured noise v ρ , v θ The mean is 0 and the covariance is δ, respectively. ρ 2 , δ θ 2 Gaussian white noise; ρ(k) is the radial distance of the target measured by the millimeter-wave radar at time k; θ(k) is the direction angle of the target measured by the millimeter-wave radar at time k;

[0046] S42. Since the extended Kalman filter in the KCF target tracking algorithm can only track one target at a time, a data association method needs to be introduced to enable the millimeter-wave radar to track multiple targets simultaneously.

[0047] Since the data association algorithm uses centroid data based on clustering and reference processes, it first calculates the distance between each target (old centroid) and the observation (new centroid) in the polar coordinate system (i.e., the millimeter-wave radar coordinate system); finds the globally minimum distance from the above distances, and associates the tracked target with the new centroid linked by this distance; until all unassociated new centroids and tracked targets have been associated, the association relationship is output.

[0048] S43. The associated centroid is passed to the KCF target tracking algorithm so that the update phase of the extended Kalman filter can complete the new state estimation, thereby enabling the millimeter-wave radar to track multiple targets in real time.

[0049] Furthermore, the specific processing procedure for step S5 is as follows:

[0050] S51. The image sequence with radar point projection is split into two paths and fed into the target detection thread and the target tracking thread. In the target detection thread, the target detection box is finally obtained. In the target tracking thread, the target in the detection box is used as a training sample to initialize the tracking box in the target tracking algorithm.

[0051] S53. Subsequent image samples and training samples are fed into the kernel correlation filter of the target tracking algorithm for correlation operation (i.e., a kernel correlation filter is trained using the training samples, and the filter is slid across the subsequent image samples to obtain the filter response value at each position). The peak value of the filter response MAX_RES is compared with a set threshold. If it is higher than or equal to the set threshold, the target tracking is considered successful. At this time, the peak value of the response is used as the position of the new tracking box, and the target tracking box is updated iteratively. If it is lower than the set threshold, the tracking is considered to have failed, and the process jumps to the target tracking thread.

[0052] In summary, this invention employs the aforementioned autonomous driving target detection and tracking method. It utilizes radar positioning information to assist the camera target detection algorithm, providing prior information and candidate weights for the potential locations of targets in the input image of the YOLOv5 algorithm. Simultaneously, it introduces the SENET channel attention mechanism module into the YOLOv5 algorithm, allowing the algorithm to apply different attention to image channels of varying importance during operation, thereby improving the speed and accuracy of road target detection. By fusing the YOLOv5 target detection algorithm with the KCF target tracking algorithm, real-time camera tracking of targets is achieved. Since the detection bounding boxes output by the detection algorithm can be used to initialize the KCF target tracking algorithm, the overall stability of camera target tracking is improved. When a target in the camera is occluded or the camera tracking effect is unsatisfactory, the system jumps to a radar tracking thread combining extended Kalman filtering and data association, effectively improving the stability of target tracking in intelligent vehicles. Attached Figure Description

[0053] Figure 1 This is an overall flowchart of the target detection and tracking process of the present invention;

[0054] Figure 2 This is a schematic diagram illustrating the construction principle of the channel-space attention mechanism module in this invention.

[0055] Figure 3 This is a model diagram of the improved YOLOv5 target detection algorithm in this invention;

[0056] Figure 4 This is a model diagram of the KCF target tracking algorithm in this invention;

[0057] Figure 5 This is a schematic diagram of the camera target tracking principle that integrates YOLOv5 and KCF algorithms in this invention;

[0058] Figure 6 These are images acquired by the method of the present invention during normal vehicle operation;

[0059] Figure 7 This is an image captured by the method of the present invention during the turning of the vehicle in front. Detailed Implementation

[0060] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0061] This invention employs the YOLOv5 detection framework with several improvements. A radar feature map spatial attention module is constructed to provide the target's position weight matrix in the camera image. A SENET channel attention module is introduced to provide the weight matrix for each image channel. Finally, the radar spatial attention module and the SENET channel attention module are combined to form an attention mechanism module for target detection. This attention mechanism module is then added between the backbone and neck layers of the YOLOv5 detection framework, forming a target detection network that fuses radar information with an attention mechanism. This effectively improves the speed and accuracy of target detection in intelligent vehicles.

[0062] This invention employs a multi-threaded tracking framework combining camera and radar. In the camera tracking section, it integrates YOLOv5 target detection and KCF target tracking algorithms, fully leveraging the advantages of online training of the KCF algorithm. The detection boxes output by the YOLOv5 detection algorithm are directly input into the KCF tracking algorithm for initialization, enabling iterative and continuous target tracking. When camera tracking is unsatisfactory due to rain, fog, or target obstruction, the tracking switches to the radar tracking thread. In the radar tracking section, an Extended Kalman Filter (EKF) combined with data association methods enables simultaneous radar tracking of multiple road targets.

[0063] Specifically, such as Figure 1 The autonomous driving target detection and tracking method shown includes the following steps:

[0064] S1. Perform time and space registration on the data collected by the vehicle-mounted millimeter-wave radar and the camera, and then project the millimeter-wave radar points into the camera image coordinate system through coordinate transformation.

[0065] S2. Construct radar feature maps and leverage the strong positioning capabilities of radar to provide a spatial attention module for camera target detection algorithms.

[0066] S3. The SENET channel attention module and the radar feature map spatial attention module are introduced into the backbone structure of the target detection algorithm YOLOv5, so that the target detection algorithm pays more attention to the area where the target may exist, thereby improving the speed and accuracy of target detection.

[0067] S4. Construct a KCF target tracking algorithm that integrates extended Kalman filtering and data association algorithms;

[0068] S5. When the camera is not obstructed, the target detection algorithm and the target tracking algorithm are integrated to track the vehicle target; when the camera is obstructed, the target tracking algorithm is used to track the vehicle target.

[0069] In step S1, since the sensors need to be synchronized in time (i.e., different sensors need to collect information at the same time), the time reference should be the sensor with the lowest frequency. Because the radar has a low acquisition frequency, the radar data acquisition frequency should be used as the reference.

[0070] To transfer the target detected by millimeter-wave radar onto the image captured by the camera, it is necessary to complete the coordinate transformation between the millimeter-wave radar coordinate system, the 3D world coordinate system, the camera coordinate system, the image coordinate system, and the pixel coordinate system. First, the millimeter-wave radar coordinate system should be transformed to a world coordinate system centered on the camera position.

[0071]

[0072] Then transform the world coordinate system into the camera coordinate system.

[0073]

[0074] Then transform the camera coordinate system into the image coordinate system.

[0075]

[0076] Then transform the image coordinate system into the pixel coordinate system.

[0077]

[0078] Based on formulas (1)-(4), the transformation from the world coordinate system to the pixel coordinate system can be achieved, that is...

[0079]

[0080] Finally, the spatial transformation model from millimeter-wave radar to camera can be obtained.

[0081]

[0082] In the formula, [x r y r z r 1] T Here are the coordinates of the target in millimeter-wave radar coordinates, and their corresponding pixel coordinates are [x...]. c y c z c 1] T ;[uv 1] TLet R be the target coordinates in pixel coordinates; R is a 3×3 identity orthogonal rotation matrix; T is a 3×1 translation matrix; I is the identity matrix; d x With d y These represent the size of each pixel in physical units on the horizontal and vertical axes, respectively; f is the camera's focal length; k is the scaling factor; x r y r z r Let u and v be the three-dimensional coordinates of the target in the millimeter-wave coordinate system; let z be the two-dimensional coordinates of the target in the pixel coordinate system; z is the three-dimensional coordinates of the target in the millimeter-wave coordinate system. c The coordinates are in the camera coordinate system; u0 and v0 are the two-dimensional coordinates in the pixel coordinate system before the coordinate system rotation transformation operation.

[0083] Furthermore, in step S2, since the target detection framework uses the YOLOv5 framework and processes information in image or video format, integrating millimeter-wave radar information into the target detection algorithm requires representing the radar information in image format. Therefore, the radar points are first transformed into the image coordinate system, and then a blank single-channel image of the same size as the original image is generated. All pixels in the image are set to 0, and then a Region of Interest (ROI) of a certain area is generated centered on the radar points in the image. The formula for the ROI area is:

[0084] S=αn+β (7)

[0085] In the formula, n is the number of radar reflection points; α and β are hyperparameters.

[0086] α and β can be learned by maximizing the IOU between the ROI and the target ground truth bounding box, as shown in the formula:

[0087]

[0088] In the formula, N is the number of input samples during the hyperparameter learning process; M i The number of bounding boxes for the object in the i-th sample.

[0089] The formula for calculating IOU is:

[0090]

[0091] In the formula, ROI represents the ROI generated by the radar point; GT represents the target's true bounding box; and S represents the area to be calculated.

[0092] After determining the ROI area, set the pixel value to 1 within the generated ROI area, and finally obtain a radar feature map with the pixel value of 1 within the ROI area of ​​the radar point and the pixel value of 0 in other areas.

[0093] The radar feature map is essentially a weighted matrix representing the position of a target in an image. By multiplying the camera image with the radar feature map, the target detection algorithm can suppress pixel values ​​at locations where the target does not exist, thus allowing the algorithm to focus on locations where the pixel value is 1, i.e., where the radar may detect the target.

[0094] Furthermore, in step S3, SENET is a network structure that, unlike the radar feature maps mentioned above, focuses on the channel dimension rather than the spatial dimension, helping the target detection algorithm concentrate on effective targets. This network improves performance by modeling the channel relationships in the image, which includes two key operations: Squeeze and Excitation, ultimately revealing the interdependencies between the modeled feature channels.

[0095] In SENET, the Squeeze operation compresses features along the spatial dimension, transforming each two-dimensional feature channel into a real number and matching the output dimension to the number of input feature channels. This represents the global distribution of responses across feature channels and allows layers closer to the input to also gain a global receptive field. The Excitation operation, similar to the gate mechanism in recurrent neural networks, generates weights for each feature channel through parameters, which are learned to explicitly model the correlations between feature channels. Finally, the weights output by Excitation are interpreted as the importance of each feature channel after feature selection, and then multiplied and weighted channel-wise onto each channel of the input image.

[0096] In summary, SENET learns to automatically determine the importance of each channel in an image, and then uses this importance to enhance features that are useful for the current task while suppressing features that are not useful for the current task.

[0097] Therefore, in the above steps, we constructed a radar feature map representing the target location weights and described the SENET network structure representing the weights of different channels of the camera image. We combined the two to form a channel-spatial attention module, which is represented as follows: Figure 2 As shown. Adding this channel-spatial attention module to the backbone structure of the YOLOv5 object detection algorithm forms a YOLOv5 object detection algorithm based on the channel-spatial attention mechanism. The structure of this object detection algorithm is as follows. Figure 3 As shown.

[0098] Furthermore, in step S4, since millimeter-wave radar measures based on the Doppler effect, the data measured by the millimeter-wave radar is the target's range, velocity, and azimuth angle in polar coordinates. Because the Kalman filter algorithm is only applicable to linear systems, and millimeter-wave radar involves a nonlinear process of converting from polar coordinates to Cartesian coordinates during state updates, Kalman filtering cannot be used to track the millimeter-wave radar in this case. The extended Kalman filter (EKF) principle is similar to the Kalman filter, but the difference lies in that the EKF performs a Taylor series expansion and first-order linearization truncation on the measurement matrix. Therefore, we use the EKF extended Kalman filter algorithm to track the millimeter-wave radar.

[0099] Since the EKF algorithm (Extended Kalman Filter) can only track a single target, while intelligent vehicles may encounter multiple targets in a road driving environment, data association methods are combined with extended Kalman filtering to meet the real-time tracking requirements of millimeter-wave radar for multiple targets. The principle of the millimeter-wave radar target tracking algorithm combining EKF and data association is as follows: Figure 4 As shown.

[0100] Furthermore, in step S5, the target detection part of this paper adopts a camera-based, radar-assisted strategy, and this strategy is also used in the target tracking part. When the camera tracking effect is poor under conditions such as extreme rain or fog with low visibility, or when the target is obscured, radar tracking is switched to the camera tracking method. The camera tracking method integrates the YOLOv5 target detection and KCF target tracking algorithms.

[0101] The advantage of the YOLOv5 algorithm lies in its deep convolutional network for extracting target features and its multi-scale fusion detection strategy of FPN+PAN, which improves the accuracy and robustness of target detection. However, deep learning-based detection algorithms have high requirements for training samples. If the current sample differs significantly from the training sample, the algorithm may fail to detect the target, leading to tragedies such as a Tesla self-driving car colliding with a white truck. Furthermore, the algorithm cannot continuously and stably track the target when it cannot detect it. The KCF tracking algorithm, on the other hand, uses an online learning training strategy, eliminating the need for pre-training a large number of samples. During tracking, a filter is trained based on the current frame of the video. This filter is used to determine the target's position in the next frame, and the filter is updated with the new target position. This iterative process is repeated to achieve target tracking. Therefore, by outputting the target detection box detected by the YOLOv5 algorithm to the KCF tracking algorithm, the camera can continuously track the target. The principle of the entire tracking system is as follows: Figure 5 As shown, the images acquired during the detection phase are as follows: Figure 6 As shown, the images acquired during the tracking phase are as follows: Figure 7 As shown.

[0102] The above are specific embodiments of the present invention, but the scope of protection of the present invention should not be limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. An automatic driving target detection and tracking method based on multi-source heterogeneous information fusion, characterized in that, S1, time and space registration is performed on the data collected by the vehicle-mounted millimeter wave radar and the camera, and then the millimeter wave radar points are projected into the camera image coordinate system through coordinate transformation; S2, a radar feature map is constructed to provide a spatial attention module for a camera target detection algorithm; S3, the SENET channel attention module and the radar feature map spatial attention module are introduced into the backbone structure of the target detection algorithm YOLOv5 to improve the speed and accuracy of target detection; S4, a KCF target tracking algorithm is constructed by fusing the extended Kalman filter and the data association algorithm; S5, under the condition that the camera is not blocked, the vehicle target is tracked by fusing the target detection algorithm and the target tracking algorithm; under the condition that the camera is blocked, the vehicle target is tracked based on the target tracking algorithm.

2. The automatic driving target detection and tracking method according to claim 1, characterized in that, The specific processing process of step S1 is as follows: S11, time registration of the two sensors is realized by using the radar sampling to trigger camera sampling; S12, space registration of the millimeter wave radar and the camera is realized by using a space conversion model; The space conversion model is expressed as follows: In the formula, [x r y r z r 1] T is the coordinate of the target in the millimeter wave radar coordinate system;[u v 1] T is the coordinate of the target in the pixel coordinate system; and R is a 3x3 unit orthogonal rotation matrix. T is a 3x1 translation matrix; I is an identity matrix; d x with d y representing the size of each pixel in physical units along the horizontal and vertical axes, respectively; f is the focal length of the camera; k is the scaling factor; x r , y r , z r are the three-dimensional coordinates of the target in the millimeter wave radar coordinate system; u, v are the two-dimensional coordinates of the target in the pixel coordinate system; z c is the coordinate in the camera coordinate system; u0, v0 are the two-dimensional coordinates in the pixel coordinate system before the coordinate system rotation transformation operation. S13, based on the registered data, the millimeter wave radar points are projected into the camera image. 3.The automatic driving target detection and tracking method of claim 2, wherein, The specific processing process of step S2 is as follows: S21, a blank single-channel radar feature map with the same size as the image is generated, and the initial value of each pixel point in the radar feature map is set to 0; S22, after initializing the radar feature map with pixel point values of 0, an ROI region of interest is constructed with a single radar projection point in the radar feature map as the center, i.e. the possible position of the target detected by the radar, and then the pixel point values in the ROI region of interest of the radar feature map are set to 1; S23, the area of the ROI region of interest is calculated according to the number of clustered radar points, and the calculation formula is as follows: S=αn+β In the formula, n is the number of radar reflection points; and α and β are hyperparameters; α and β are obtained by learning to maximize the IOU of the ROI and the real labeled box, and the formula is as follows: In the formula, N is the number of input samples in the learning hyperparameter process; M i is the number of labeled boxes of objects in the i-th sample; The calculation formula of IOU is as follows: In the formula, ROI represents the ROI region of interest; GT represents the real labeled box region; and S represents the area: S24, finally, a radar feature map representing target position information is generated, in which the pixel values in the ROI region of interest are 1 and the pixel values in other parts are 0.

4. The automatic driving target detection and tracking method according to claim 3, characterized in that, The specific processing process of step S3 is as follows: S31, the channel weights of different channels of the input feature map are calculated based on the SENET channel attention; S32, the SENET channel attention module is added to the last layer of the backbone structure of the target detection algorithm YOLOv5, and the feature map is processed according to the channel weights; S33, the feature map processed by the SENET channel attention module is multiplied by the radar feature map, so that the feature map is affected by both the channel attention weight and the spatial position attention weight, thereby constructing a channel-spatial attention mechanism module that fuses radar information; S34, insert the channel-space attention module into the last layer of the backbone structure of the target detection algorithm YOLOv5 to obtain the candidate region and prior information where the target may exist.

5. The automatic driving target detection and tracking method according to claim 4, characterized in that, The specific processing process of step S4 is as follows: S41, in the radar detection coordinate system, the bicycle dynamics model is used to model the vehicle target, and the relative motion equation between the host vehicle and the front vehicle target is obtained The relative motion equation between the host vehicle and the front vehicle target is obtained In the formula, a, b are input quantities of the model; B is the wheelbase of the preceding vehicle; v is the relative speed of the host vehicle and the preceding vehicle; is the front wheel steering angle of the preceding vehicle; θ is the direction angle of the preceding vehicle; x r , y r are the current coordinates of the vehicle; Discretize the above formula, then: where δ a where δ b is the disturbance noise; δ P where δ s is the multiplication noise, which represents the increase of the vehicle motion state uncertainty when the vehicle steering angle and speed change; the above four noises are mutually independent and are Gaussian white noises with mean 0 and known covariance; The measurement equation of the millimeter wave radar at time k is: wherein the measurement noise v ρ , v θ are Gaussian white noises with mean 0 and covariance δ ρ 2 , δ θ 2 respectively; p(k) is the radial distance of the target measured by the millimeter wave radar at time k; θ(k) is the direction angle of the target measured by the millimeter wave radar at time k; S42, introduce a data association method to enable the millimeter wave radar to track multiple targets simultaneously; In the radar coordinate system, find the new particle with the smallest global distance from the old particle and associate it with the tracked target; Until all unassociated new centroids and tracked targets have been associated, output the association relationship; S43, pass the associated centroids to the KCF target tracking algorithm to complete the new state estimation in the update stage of the extended Kalman filter, so that the millimeter wave radar can track multiple targets in real time.

6. The automatic driving target detection and tracking method according to claim 5, characterized in that, The specific processing process of step S5 is as follows: S51, divide the image sequence projected by the radar points into two paths to enter the target detection thread and the target tracking thread; in the target detection thread, the detection box of the target is finally obtained; In the target tracking thread, the target in the detection box is taken as a training sample to initialize the tracking box in the target tracking algorithm; S53, put the subsequent image samples and the training samples into the kernel correlation filter of the target tracking algorithm, and compare the filter response peak value MAX_RES with the set threshold value; if it is higher or equal to the set threshold value, it is considered that the tracking target is successful, at this time the response peak value point is taken as the position of the new tracking box, and the target tracking box is iteratively updated; if it is lower than the set threshold value, it is considered that the tracking fails, and the target tracking thread is jumped to.

Images