Obstacle detection method and device based on multi-millimeter-wave radar data

Abstract

The invention discloses an obstacle detection method and system based on multi-millimeter-wave radar data, and relates to obstacle detection. The method comprises the following steps: collecting multi-path millimeter-wave radar data; carrying out FFT (Fast Fourier Transform) on each path of radar data, extracting radar point cloud data, and marking a radar ID (Identity) and a timestamp of each point cloud data to obtain a multi-source point cloud data set with identification information; performing coordinate transformation on the multi-source point cloud data set with the identification information according to the installation position and angle of each millimeter wave radar in the vehicle body coordinate system to obtain a point cloud data set of spatial registration; performing fusion processing on redundant point cloud data generated by repeated detection of the same obstacle by a plurality of radars to obtain fusion point cloud data after redundancy elimination; a data association algorithm is adopted to match the fused point cloud data at the continuous moments, and a filtering algorithm is utilized to predict and update the motion state of the obstacle to obtain a dynamic target tracking result containing the position and speed of the obstacle; obstacles around the vehicle can be detected.

Description

Technical Field

[0001] This application relates to the field of obstacle detection, and in particular to an obstacle detection method and apparatus based on multi-millimeter-wave radar data. Background Technology

[0002] With the advancement of intelligent mine construction, automatic driving technology for underground mine transportation equipment has become one of the key technologies for improving coal mine safety and transportation efficiency. However, the unique operating environment of underground mines poses severe challenges to vehicle perception systems. Narrow underground tunnels, dim lighting, and the presence of large amounts of water mist and dust severely restrict the application effectiveness of traditional perception technologies.

[0003] Current perception solutions for underground mining transport vehicles primarily employ a multi-sensor fusion architecture, with lidar as the main sensor and cameras and millimeter-wave radar as supplementary sensors. Specifically, the main lidar is typically mounted on the top of the vehicle, responsible for perceiving the vehicle's surrounding environment; the camera is mounted at the front of the vehicle, working in conjunction with the lidar to identify obstacles ahead; and the millimeter-wave radar is mounted on the left and right front sides of the vehicle, supplementing the lidar and detecting whether it is safe to pass on either side. This sensor data is processed through point cloud-level fusion (front fusion) or target-level fusion (back fusion).

[0004] However, the aforementioned traditional sensing solutions have many limitations in underground mining environments: First, lidar faces severe performance degradation in underground environments. Water mist discharged from underground tunnels causes laser scattering, resulting in numerous false detections; dusty environments absorb and scatter laser signals, significantly reducing detection range and accuracy; simultaneously, lidar is expensive, and the massive point cloud data it generates requires high-performance computing resources for real-time processing. Second, cameras are almost ineffective in dark underground environments. Even with supplemental lighting, the camera's sensing performance is extremely unstable under complex lighting conditions of alternating strong and weak light. Vision-based deep learning algorithms suffer from false detections and false negatives, and require a large amount of labeled data for training, with the model's decision-making process lacking interpretability. Third, although millimeter-wave radar has the natural advantage of penetrating water mist and dust, and can maintain stable detection performance in harsh environments, the detection angle of a single millimeter-wave radar is typically only ±60°, far from meeting the need for 360° all-around obstacle detection around vehicles.

[0005] Furthermore, multi-sensor data fusion faces technical challenges such as data redundancy and conflicts, high algorithm complexity, and stringent real-time requirements. The differences in noise characteristics and uncertainties among different sensors increase the difficulty of designing fusion algorithms.

[0006] Therefore, how to fully leverage the anti-interference advantages of millimeter-wave radar in harsh underground environments and overcome the physical limitations of the detection angle of a single radar through the coordinated operation of multiple millimeter-wave radars to achieve highly reliable 360° obstacle detection around vehicles in narrow tunnels of underground mines has become an urgent technical challenge. Summary of the Invention

[0007] To address the limitations of traditional lidar and camera sensing caused by water mist, dust, and darkness in narrow mine tunnels, as well as the inability of a single millimeter-wave radar to perform 360° real-time obstacle detection of vehicles due to its limited detection angle, this application provides an obstacle detection method and device based on multi-millimeter-wave radar data. By coordinating the deployment of eight millimeter-wave angular radars and data fusion processing, the limitation of the physical detection angle of a single radar is overcome, enabling real-time 360° obstacle detection around vehicles in narrow mine tunnel environments.

[0008] One aspect of this application provides an obstacle detection method based on multi-millimeter-wave radar data, comprising: S1, parallel acquisition of multiple millimeter-wave radar data streams; S2, performing FFT transformation on each radar data stream to extract radar point cloud data, and marking the radar ID and timestamp of each point cloud data to obtain a multi-source point cloud dataset with identification information; S3, performing coordinate transformation on the multi-source point cloud dataset with identification information according to the installation position and angle of each millimeter-wave radar in the vehicle coordinate system, converting the point cloud data in the local coordinate system of each radar to the vehicle coordinate system, to obtain a spatially registered point cloud dataset; S4, identifying obstacles located in the phase according to the radar ID and the three-dimensional coordinates of the point cloud in the vehicle coordinate system. The point cloud data of the overlapping detection area of ​​adjacent radars is fused to obtain redundant point cloud data generated by repeated detection of the same obstacle by multiple radars, resulting in redundancy-free fused point cloud data; S5, according to the timestamp, the fused point cloud data of consecutive time moments are matched using a data association algorithm to establish the obstacle target trajectory, and the motion state of the obstacle is predicted and updated using a filtering algorithm to obtain dynamic target tracking results containing the obstacle position and velocity; S6, based on the redundancy-free fused point cloud data and the dynamic target tracking results, the obstacle information is mapped onto a preset grid map, and the occupancy probability of each grid is calculated through probability update rules to obtain a real-time updated 360° obstacle grid map.

[0009] Furthermore, S1, multiple millimeter-wave radar data are collected in parallel, including: 8 millimeter-wave radar data are collected simultaneously by 4 millimeter-wave angle radars arranged at the front, rear, left, and right of the vehicle, and 4 millimeter-wave angle radars set at the left front, right front, left rear, and right rear; wherein, the detection areas of adjacent millimeter-wave angle radars are set with an overlap angle of 15° to 30°, so that the detection range of the 8 millimeter-wave radars together covers a 360° range around the vehicle; the millimeter-wave angle radars use the 77GHz or 79GHz frequency band.

[0010] In this scheme, millimeter-wave angular radar refers to a millimeter-wave radar capable of simultaneously acquiring four-dimensional information about a target: range, azimuth, elevation, and velocity. Compared to traditional 3D millimeter-wave radar, 4D radar adds elevation angle detection capability, enabling more accurate location of obstacles at the top and bottom of underground tunnels, which is particularly important in narrow tunnel environments with limited height.

[0011] The 360° range around the vehicle refers to the all-around detection area centered on the underground mining transport vehicle. Due to the narrowness of underground roadways and the presence of complex structures such as branch roads and chambers, vehicles may encounter obstacles from all directions when driving, turning, or reversing. Therefore, it is necessary to achieve no blind spots in front of, behind, to the left, to the right, and in all diagonal directions to ensure safe operation in confined spaces.

[0012] Further, in step S2, an FFT transformation is performed on each radar data stream to extract radar point cloud data. Each point cloud data point is then labeled with its corresponding radar ID and timestamp, resulting in a multi-source point cloud dataset with identification information. This includes: adaptive threshold detection of the millimeter-wave radar data to obtain denoised radar data; an FFT transformation is performed on the denoised radar data to obtain the amplitude and phase of N frequency components, where N is the number of FFT points; and according to the radar equation f = 2vr / λ, the frequency components are converted into corresponding range information, where f is the frequency, v is the electromagnetic wave velocity, and r is the target range. Using wavelength as the reference wavelength, distance distribution data of obstacle targets is obtained. Based on the millimeter-wave angle radar, multiple receiving antennas built into it are used to acquire multi-channel received signals of the same target. For distance points with amplitudes exceeding a preset threshold, the phase information of the distance point in each receiving antenna channel is extracted, and the phase difference between adjacent antenna channels is calculated. and according to Calculate the target azimuth angle θ, where d is the distance between adjacent antennas;

[0013] The distance r, azimuth angle θ, and amplitude A of each obstacle target are combined to form point cloud data. Add the radar ID number that generated the data and the timestamp 't' when the data was collected to each point cloud data point, resulting in the format: A multi-source point cloud dataset with identification information.

[0014] Among them, adaptive threshold detection is a dynamic noise suppression technology designed for downhole water mist and dust environments. By calculating the statistical characteristics (mean μ and standard deviation σ) of the echo data in real time, it automatically adjusts the detection threshold (μ+kσ), which can effectively filter out weak reflection clutter generated by water mist and dust particles, while retaining the echo signal of the real obstacle, thus improving the detection reliability in harsh environments.

[0015] The frequency domain information obtained after FFT transformation, where N is the number of FFT points, determines the distance resolution. In the underground environment, obstacles at different distances (such as tunnel walls, support equipment, and other vehicles) will generate echoes of different frequencies. By analyzing the amplitude of these frequency components, the presence of obstacles can be determined, and the precise location of the target can be further calculated using phase information.

[0016] Through radar equations The calculated radial distance from the target to the radar is crucial for determining safe distances between vehicles and the tunnel walls, as well as between vehicles in front and behind, in narrow passageways. It is the fundamental data for avoiding collisions.

[0017] Millimeter-wave angle radar incorporates multiple receiving antennas to simultaneously receive electromagnetic wave signals reflected from the same target. By comparing the phase differences of the signals received by different antennas, the target's angular information can be calculated. In the confined space of underground mines, the multi-channel design improves angular resolution, enabling the differentiation of multiple nearby obstacles.

[0018] Adjacent antenna channels refer to the signal channels formed by two physically adjacent receiving antennas within the radar. The signals received by adjacent channels have a fixed phase difference relationship. This phase difference is directly related to the target angle and is the physical basis for angle measurement.

[0019] The spacing between adjacent antennas refers to the physical distance *d* between two adjacent receiving antennas. This parameter directly affects the accuracy of angle measurement and the maximum unambiguous angle range. In downhole applications, a reasonable antenna spacing design can ensure angular resolution while avoiding angular ambiguity within the radar's limited detection angle (±60°).

[0020] Furthermore, adaptive threshold detection is performed on the millimeter-wave radar data to obtain denoised radar data, including: calculating the mean amplitude μ and standard deviation σ of the echo data of the current frame; setting the detection threshold to μ+kσ, where k is a constant of 3~5; setting the echo data with amplitude below the detection threshold to zero to obtain denoised radar data, so as to filter out weak reflection clutter generated by downhole water mist and dust;

[0021] In this scheme, echo data refers to the signal data received by the radar after the electromagnetic waves emitted by the millimeter-wave radar encounter various objects in the underground environment (including tunnel walls, support equipment, transport vehicles, water mist, and dust particles). In the underground environment, echo data contains useful obstacle reflection signals and a large amount of environmental interference signals. Echoes from real obstacles are strong and stable, while echoes from water mist and dust are weak and randomly distributed. These two types of echoes can be effectively distinguished using statistical characteristics (mean μ and standard deviation σ).

[0022] Furthermore, S3, obtains the spatially registered point cloud dataset, including: acquiring the installation parameters of each millimeter-wave radar in the vehicle coordinate system, including: installation position coordinates. and installation angle ,in, This is the translation vector of the radar relative to the origin of the vehicle coordinate system. These represent the radar's rotation angles around the X, Y, and Z axes of the vehicle's coordinate system; based on the radar ID of each point cloud data point in the multi-source point cloud dataset with identification information, the installation parameters of the corresponding radar are retrieved; the point cloud data... polar coordinates in Convert to Cartesian coordinates in the radar local coordinate system, where, , , According to the installation angle Calculate the rotation matrix R from the radar local coordinate system to the vehicle coordinate system;

[0023] Through coordinate transformation formula The point cloud coordinates in the radar local coordinate system are transformed to the vehicle coordinate system; the amplitude A, radar ID, and timestamp t of the point cloud are preserved, resulting in the following format in the vehicle coordinate system: Spatial registration point cloud dataset.

[0024] Further, in S4, the redundancy-free fused point cloud data is obtained, including: determining the boundary of the overlapping detection area of ​​each adjacent radar pair based on the overlap angle of the detection areas of adjacent millimeter-wave angular radars; and determining the boundary of the overlapping detection area of ​​each adjacent radar pair based on each point cloud data in the spatially registered point cloud dataset. Using coordinates Determine whether the corresponding point cloud is located within the overlapping detection area; for point cloud data located within the overlapping detection area, calculate the distance between point clouds with different radar IDs. ; distance Less than the preset threshold The point cloud is identified as repeated detection points for the same obstacle, among which, Based on the range resolution setting of the millimeter-wave radar, multiple duplicate detection points identified as the same obstacle are fused; the maximum amplitude value and earliest timestamp of the fused point cloud are retained, duplicate point clouds are deleted, and redundancy-free fused point cloud data is obtained.

[0025] The overlapping detection area boundary refers to the boundary line where the detection ranges of two adjacent millimeter-wave radars intersect. Due to the detection angle limitation of a single millimeter-wave radar (±60°), eight radars need to be overlapped by 15°-30° to achieve 360° full coverage. Accurate determination of the overlapping detection area boundary is crucial for identifying which point cloud data might be detected simultaneously by multiple radars. In narrow tunnel environments, large obstacles such as tunnel walls are very likely to be within the overlapping area; accurately delineating the boundary can prevent the same tunnel wall from being misidentified as multiple independent obstacles.

[0026] Preset threshold This is the distance standard for determining whether two point clouds belong to the same obstacle; its value is set based on the range resolution of millimeter-wave radar. In narrow underground tunnels, due to limited space and densely distributed obstacles, if... Setting the value too high may misclassify adjacent obstacles (such as side-by-side support columns) as the same obstacle; setting it too low may misclassify different parts of the same obstacle (such as the front and rear ends of large equipment) as multiple obstacles. A reasonable setting is needed. The value is usually set to 2-3 times the radar range resolution, which can effectively merge repeated detection points of the same obstacle while maintaining the ability to distinguish dense obstacles.

[0027] Furthermore, in step S5, dynamic target tracking results containing obstacle position and velocity are obtained, including: dividing the fused point cloud data into consecutive data frames in chronological order based on the timestamp t in the deredundant fused point cloud data; for each point cloud in the current frame, according to the corresponding coordinates... The algorithm searches for candidate matching points in the previous frame's point cloud data whose distance is less than an association threshold, calculated based on the time interval between adjacent frames and the vehicle's maximum speed. A nearest neighbor data association algorithm is used to associate the current frame's point cloud with the nearest candidate matching point from the previous frame, establishing a cross-frame point cloud correspondence. Based on the successfully associated point cloud sequence, obstacle target trajectories are constructed. Each trajectory includes: obstacle target ID, historical position sequence, and timestamp sequence. A Kalman filter algorithm is used to estimate the state of each obstacle target trajectory, where the state vector includes the obstacle target's position (x, y, z) and velocity. The prediction step estimates the current position based on the state at the previous time step, and the update step uses the current observations to correct the prediction results.

[0028] Output the ID, position (x, y, z), and velocity of each tracked obstacle target. This yields dynamic target tracking results containing obstacle position and velocity; the association threshold refers to the maximum allowable distance threshold for determining whether two point clouds at adjacent time points belong to the same moving obstacle. This threshold is based on the time interval between adjacent frames. Maximum speed of underground vehicles The dynamic calculation yields the following formula: Related Threshold: .

[0029] In the narrow tunnel environment of underground mines, vehicle speeds are typically limited to 5–10 km / h due to the narrowness of the tunnels, resulting in a relatively small association threshold. Multiple slowly moving targets may exist within the confined space (such as transport vehicles moving in front and behind, or workers walking), and an excessively large association threshold can lead to incorrect association of different targets. Underground vehicles require frequent starts, stops, and turns, and the association threshold needs to adapt to this non-uniform motion characteristic. A reasonable association threshold can ensure continuous tracking of the same target while avoiding the misassociation of nearby moving targets.

[0030] In this scheme, the nearest neighbor data association algorithm is used to determine which point cloud in the previous frame corresponds to the same target as the obstacle point cloud detected in the current frame. Among all candidate matching points that satisfy the association threshold constraint, the point with the smallest Euclidean distance is selected as the matching result.

[0031] In the narrow tunnel environment of underground mines, the nearest neighbor algorithm has low computational complexity, meets the requirements of real-time processing underground, and is suitable for embedded system deployment. In narrow tunnels, the movement trajectory of obstacles is relatively simple (mainly along the tunnel direction), and the nearest neighbor principle can usually correctly associate them. For stationary obstacles (such as tunnel walls and support equipment), the nearest neighbor algorithm can track them stably. When multiple moving targets are close to each other (such as vehicles meeting), with reasonable association thresholds, different targets can still be effectively distinguished. The algorithm is robust and can maintain the continuity of target tracking even when water mist and dust cause partial loss of point cloud.

[0032] Furthermore, obtaining dynamic target tracking results that include obstacle position and velocity also includes: for target trajectories that have not been updated for multiple consecutive frames, determining that the target has disappeared and deleting the corresponding trajectory. In particular, underground narrow tunnels are a highly dynamic working environment, with obstacles such as transport vehicles, personnel, and mobile equipment frequently entering and leaving the radar detection range. If departing target trajectories are not deleted in time, obstacle ghosting will occur—the system still believes an obstacle exists at a certain location, but in reality, the obstacle has been moved. In narrow tunnels, this erroneous information is extremely harmful: it may cause vehicles to mistakenly judge areas that are passable as impassable, resulting in unnecessary stops and even blockages of the entire tunnel. By setting a rule to delete obstacles that have not been updated for multiple consecutive frames, the system ensures that only truly existing obstacle information is retained.

[0033] Furthermore, in S6, a real-time updated 360° obstacle grid map is obtained, including: creating a grid map centered on the origin of the vehicle coordinate system, setting the grid resolution to ensure the map covers a 360° area around the vehicle; and retrieving the coordinates of each point from the deredundant fused point cloud data. Map the grid to the corresponding grid index in the grid map; calculate the grid occupancy probability increment based on the point cloud data mapped to each grid; update the occupancy probability of each grid by using a probability update rule, combining the grid's historical occupancy probability and the current observation data; mark the grid as an obstacle grid, a free grid, or an unknown grid based on the occupancy probability and a preset threshold; output a 360° obstacle grid map containing the grid occupancy status and occupancy probability.

[0034] The grid occupancy probability increment refers to the contribution of the currently observed point cloud data to the grid occupancy status, reflecting the change in grid occupancy probability caused by a single observation. In this scheme, the calculation of the occupancy probability increment considers the following factors: the influence of point cloud amplitude A: a larger echo amplitude indicates a stronger obstacle reflection capability (such as metal equipment or tunnel walls), resulting in a larger probability increment; the influence of detection distance r: closer detection results have higher reliability, and the probability increment is inversely proportional to the distance; and the influence of point cloud density: a larger number of point clouds within the same grid indicates a greater likelihood that an obstacle actually exists at that location. The specific calculation formula can be expressed as follows: ,in, Incremental coefficient, The maximum amplitude value, The maximum detection range is n, where n is the number of point clouds within the grid.

[0035] The probabilistic update rule employs a Bayesian probabilistic update method, fusing the historical occupancy probability of the grid with the current observation results. In a downhole environment, this update rule effectively handles sensing uncertainties: the update formula is: ;in: The occupancy probability before the update (prior probability); The likelihood probability represents the probability of observing the current data when the grid is occupied; This represents the updated occupancy probability (posterior probability). To adapt to the downhole environment, a forgetting factor α (0.9-0.95) needs to be introduced to prevent excessive accumulation of historical information. .

[0036] Based on the safety requirements of underground mines and the characteristics of narrow roadways, this application adopts a dual-threshold judgment strategy: obstacle threshold. Set to 0.65-0.75; when the occupancy probability... At that time, it is judged as an obstacle grid; considering the principle of safety first in the well, this threshold setting is relatively conservative. Free Threshold Set to 0.25 to 0.35; when the occupancy probability... When the condition is met, it is determined to be a free grid; ensure that only areas with sufficient evidence of safety are marked as passable. Unknown areas: 0.35≤P ≤0.65, areas with insufficient evidence remain unknown; these areas should be handled with caution during vehicle routing.

[0037] Special case handling: For grids that have not been updated for a long time (>5 seconds), the occupancy probability gradually returns to 0.5 (completely unknown state); for grids occupied by the vehicle itself, they are forcibly set to a free state to avoid self-blocking.

[0038] Specifically, the probabilistic grid map in this application is not a simple binary judgment of "present" or "absent," but rather reflects the probability of each area being occupied through probability values. In the narrow tunnel environment of well operations, water mist and dust can cause intermittent loss of radar signals. An obstacle may be detected in one frame but missed in the next due to environmental interference. The probability update mechanism, through the accumulation of historical information, can smooth out these detection fluctuations and provide stable environmental awareness. The occupancy probability of static obstacles (such as tunnel walls) will remain high, while the occupancy probability of a moving obstacle at its original position will gradually decrease after it passes. This mechanism allows the system to adaptively track environmental changes. Even with eight radars working together, there may still be momentary blind spots in certain extreme cases. By marking these areas as "unknown" rather than "free," the system adopts a conservative strategy to ensure safety.

[0039] Another aspect of this application provides an obstacle detection device based on multi-millimeter-wave radar data, comprising: a data acquisition module for parallel acquisition of multiple millimeter-wave radar data; a point cloud extraction module for performing FFT transformation on each radar data stream to extract radar point cloud data, and marking the radar ID and timestamp of each point cloud data to obtain a multi-source point cloud dataset with identification information; a coordinate transformation module for performing coordinate transformation on the multi-source point cloud dataset with identification information according to the installation position and angle of each millimeter-wave radar in the vehicle coordinate system, transforming the point cloud data in the local coordinate system of each radar to the vehicle coordinate system, to obtain a spatially registered point cloud dataset; and a data fusion module for fusing data based on the radar ID and the three-dimensional coordinates of the point cloud in the vehicle coordinate system. The system identifies point cloud data located in overlapping detection areas of adjacent radars, and fuses redundant point cloud data generated by repeated detection of the same obstacle by multiple radars to obtain deredundant fused point cloud data. The target tracking module matches the fused point cloud data at consecutive times based on timestamps using a data association algorithm to establish the obstacle target trajectory, and uses a filtering algorithm to predict and update the obstacle's motion state to obtain dynamic target tracking results containing the obstacle's position and velocity. The grid map module maps obstacle information onto a preset grid map based on the deredundant fused point cloud data and dynamic target tracking results, and calculates the occupancy probability of each grid through probability update rules to obtain a real-time updated 360° obstacle grid map.

[0040] Compared to existing technologies, the advantages of this application are:

[0041] This application utilizes the coordinated arrangement of eight millimeter-wave angular radars. Although each radar has a detection angle of only ±60°, the design of overlapping areas of 15° to 30° between adjacent radars and multi-source data fusion processing successfully overcomes the physical limitations of a single radar, achieving 360° all-around obstacle detection around the vehicle and completely eliminating blind spots.

[0042] In addition, by using adaptive threshold detection (μ+kσ) to specifically filter out weak reflection clutter generated by water mist and dust in the well, the natural penetrating advantage of millimeter-wave radar in the 77GHz / 79GHz band against water mist and dust is fully utilized, ensuring stable and reliable obstacle detection performance even in well environments with extremely low visibility.

[0043] Finally, the 360° obstacle grid map generated by the probability update mechanism can reflect the distribution of obstacles around the vehicle in real time. In particular, it performs reasonable probability processing on areas not covered by radar and areas that have not been updated for a long time, ensuring the accuracy and timeliness of map information in the dynamically changing underground environment. Attached Figure Description

[0044] This application will be further described by way of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting; in these embodiments, the same reference numerals denote the same structures, wherein:

[0045] Figure 1 This is an exemplary flowchart illustrating an obstacle detection method based on multi-millimeter-wave radar data according to some embodiments of this application;

[0046] Figure 2 This is an exemplary flowchart illustrating the generation of a multi-source point cloud dataset with identification information, according to some embodiments of this application;

[0047] Figure 3 This is an exemplary flowchart illustrating the redundancy removal process for fused point cloud data according to some embodiments of this application;

[0048] Figure 4 This is an exemplary flowchart illustrating the acquisition of dynamic target tracking results according to some embodiments of this application;

[0049] Figure 5 This is the target tracking code shown in some embodiments of this application. Detailed Implementation

[0050] The methods and systems provided in the embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0051] Example 1

[0052] like Figure 1 As shown, S1, multiple millimeter-wave radar data are collected in parallel, including: 8 channels of millimeter-wave radar data are simultaneously collected by 4 millimeter-wave angle radars arranged in front, rear, left, and right of the vehicle, and 4 millimeter-wave angle radars set in front left, front right, rear left, and rear right; wherein, the detection areas of adjacent millimeter-wave angle radars are set with an overlap angle of 15° to 30°, so that the detection range of the 8 millimeter-wave radars together covers a 360° range around the vehicle; the millimeter-wave angle radars use the 77GHz or 79GHz frequency band.

[0053] S2, perform FFT transformation on each radar data source to extract radar point cloud data, and label the radar ID and timestamp to which each point cloud data belongs, resulting in a multi-source point cloud dataset with identification information, including:

[0054] Adaptive threshold detection is performed on millimeter-wave radar data to obtain denoised radar data; the mean amplitude μ and standard deviation σ of the current frame echo data are calculated; the detection threshold is set to μ+kσ, where k is a constant of 3~5; echo data with amplitude below the detection threshold are set to zero to obtain denoised radar data, thereby filtering out weak reflection clutter generated by downhole water mist and dust.

[0055] Perform an FFT transform on the denoised radar data to obtain the amplitude and phase of N frequency components, where N is the number of FFT points; according to the radar equation... The frequency components are converted into corresponding distance information, where f is the frequency, v is the electromagnetic wave velocity, r is the target distance, and λ is the wavelength, thus obtaining the distance distribution data of the obstacle target.

[0056] Based on millimeter-wave angle radar, multiple built-in receiving antennas are used to acquire multi-channel received signals of the same target. For distance points with amplitudes exceeding a preset threshold, the phase information of the distance point in each receiving antenna channel is extracted, and the phase difference between adjacent antenna channels is calculated. and according to Calculate the target azimuth angle θ, where d is the distance between adjacent antennas;

[0057] The distance r, azimuth angle θ, and amplitude A of each obstacle target are combined to form point cloud data. Add the radar ID number that generated the data and the timestamp 't' when the data was collected to each point cloud data point, resulting in the format: A multi-source point cloud dataset with identification information.

[0058] like Figure 2 As shown in Figure S3, based on the installation position and angle of each millimeter-wave radar in the vehicle coordinate system, coordinate transformation is performed on the multi-source point cloud dataset with identification information to convert the point cloud data in the local coordinate system of each radar to the vehicle coordinate system, resulting in a spatially registered point cloud dataset, including:

[0059] Obtain the installation parameters of each millimeter-wave radar in the vehicle coordinate system, including: installation position coordinates. and installation angle ,in, This is the translation vector of the radar relative to the origin of the vehicle coordinate system. These are the rotation angles of the radar around the X, Y, and Z axes of the vehicle's coordinate system, respectively.

[0060] Based on the radar ID of each point cloud data in the multi-source point cloud dataset with identification information, find the installation parameters of the corresponding radar.

[0061] Point cloud data polar coordinates in Convert to Cartesian coordinates in the radar local coordinate system, where, , , ;

[0062] According to the installation angle Calculate the rotation matrix R from the radar local coordinate system to the vehicle coordinate system;

[0063] Through coordinate transformation formula The point cloud coordinates in the radar local coordinate system are transformed to the vehicle coordinate system;

[0064] Retaining the point cloud amplitude A, radar ID, and timestamp t, the following format is obtained in the vehicle coordinate system: Spatial registration point cloud dataset.

[0065] By using radar ID identification and precise coordinate system transformation, the local coordinate system data of 8 radars are unified into the vehicle coordinate system. Through overlapping area identification and distance threshold determination, repeated detection data of the same obstacle are effectively integrated, which not only ensures the integrity of detection, but also avoids data redundancy from interfering with subsequent processing.

[0066] like Figure 3 As shown, S4, based on the radar ID and the three-dimensional coordinates of the point cloud in the vehicle coordinate system, identifies the point cloud data located in the overlapping detection area of ​​adjacent radars, and fuses the redundant point cloud data generated by multiple radars repeatedly detecting the same obstacle to obtain the deredundant fused point cloud data, including:

[0067] The overlap angle of the detection areas of adjacent millimeter-wave angular radars is used to determine the boundary of the overlapping detection area of ​​each adjacent radar pair.

[0068] Based on each point cloud data in the spatially registered point cloud dataset Using coordinates Determine whether the corresponding point cloud is located within the overlapping detection area;

[0069] For point cloud data located within the overlapping detection area, calculate the distance between point clouds with different radar IDs. ;

[0070] Distance Less than the preset threshold The point cloud is identified as repeated detection points for the same obstacle, among which, Based on the range resolution of the millimeter-wave radar;

[0071] Multiple duplicate detection points identified as the same obstacle are merged;

[0072] The maximum amplitude value and earliest timestamp of the fused point cloud are retained, and duplicate point clouds are deleted to obtain the redundancy-free fused point cloud data.

[0073] like Figure 4As shown in step S5, based on the timestamp, a data association algorithm is used to match the fused point cloud data at consecutive time points to establish the obstacle target trajectory. A filtering algorithm is then used to predict and update the obstacle's motion state, resulting in a dynamic target tracking result containing the obstacle's position and velocity, including:

[0074] Based on the timestamp t in the deduplicated point cloud data, the fused point cloud data is divided into consecutive data frames in chronological order;

[0075] For each point cloud in the current frame, based on the corresponding coordinates Search for candidate matching points in the previous frame of point cloud data whose distance is less than the association threshold. The association threshold is calculated based on the time interval between adjacent frames and the maximum driving speed of the vehicle.

[0076] The nearest neighbor data association algorithm is used to associate the current frame point cloud with the nearest point among the candidate matching points in the previous frame to establish a correspondence between cross-frame point clouds;

[0077] Based on the successfully associated point cloud sequences, construct obstacle target trajectories. Each trajectory includes: obstacle target ID, historical location sequence, and timestamp sequence.

[0078] The Kalman filter algorithm is used to estimate the state of each obstacle target trajectory, where the state vector includes the obstacle target's position (x, y, z) and velocity. The prediction step estimates the current position based on the state at the previous time step, and the update step uses the current observations to correct the prediction results.

[0079] For target trajectories that have not been updated for multiple consecutive frames, determine that the target has disappeared and delete the corresponding trajectory.

[0080] Output the ID, position (x, y, z), and velocity of each tracked obstacle target. This yields dynamic target tracking results containing obstacle position and velocity. See the relevant code for details. Figure 5 ;

[0081] By using timestamp synchronization and Kalman filtering algorithms, the system can accurately track the trajectory of obstacles and predict their position and velocity in complex multipath reflection environments underground, providing reliable dynamic information support for obstacle avoidance decisions in narrow tunnels.

[0082] S6. Based on the redundancy-free fused point cloud data and dynamic target tracking results, the obstacle information is mapped onto a preset grid map. The occupancy probability of each grid is calculated through probability update rules to obtain a real-time updated 360° obstacle grid map, including: creating a grid map centered on the origin of the vehicle coordinate system, setting the grid resolution to 0.1m to 0.2m, and covering the 360° area around the vehicle to ensure that it includes the full detection range of the 8 millimeter-wave angle radars;

[0083] The coordinates of each point in the deduplicated point cloud data. Mapped to the corresponding raster index (i, j), where, , resolution refers to the raster resolution;

[0084] For a grid containing point clouds, the occupancy probability increment is calculated based on the amplitude A and distance r of the point clouds. The occupancy probability increment is directly proportional to the amplitude A and inversely proportional to the distance r.

[0085] The obstacle position (x, y, z) and velocity in the dynamic target tracking results Information is mapped onto a grid map. For moving obstacles, the grids they may occupy in the future are predicted based on their velocity vectors, and the probability of these grids being occupied is increased.

[0086] A Bayesian probability update rule is used, combining historical occupancy probabilities and current observations, to calculate the posterior occupancy probability of each grid cell. Where z is the current observation;

[0087] For blind zone grids not covered by any radar, their occupancy probability is maintained at the initial uncertainty value of 0.5; for grids that have not been updated for a long time, their occupancy probability is gradually reduced to the initial value.

[0088] Grids are classified according to their occupancy probability: grids with an occupancy probability greater than 0.7 are marked as obstacle grids, those with an occupancy probability less than 0.3 are marked as free grids, and those in between are marked as unknown grids.

[0089] The output includes a 360° obstacle grid map containing grid occupancy status, occupancy probability, and last update timestamp, enabling all-round real-time detection of obstacles around vehicles in narrow underground tunnel environments.

[0090] Example 2

[0091] A 360-degree obstacle detection device based on multi-millimeter-wave radar includes: multiple millimeter-wave radar modules distributed around the detection device, including four millimeter-wave angle radars (front, rear, left, and right) and four corner radars (left-front, right-front, left-rear, and right-rear). The detection ranges of each millimeter-wave radar module overlap, collectively forming a 360-degree detection area with no blind spots; the millimeter-wave radar modules use the 77GHz or 79GHz frequency band.

[0092] The signal processing unit is connected to the millimeter-wave radar module and is used to receive and process radar signals sent by the millimeter-wave radar module to extract obstacle information; the signal processing unit adopts an FPGA or DSP chip.

[0093] The data fusion unit, connected to the signal processing unit, is used to fuse obstacle information from different millimeter-wave radar modules to generate a unified obstacle map. The data fusion unit employs algorithms such as Hungarian filtering, Kalman filtering, and particle filtering.

[0094] The control unit, connected to the data fusion unit, is used for path planning, obstacle avoidance, and other operations based on the obstacle map. The control unit employs an embedded processor.

[0095] The foregoing illustrative description of the present application and its embodiments is not restrictive and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. The accompanying drawings are only one embodiment of the present application, and the actual structure is not limited thereto. Therefore, if those skilled in the art are inspired by this description and design similar structures and embodiments without departing from the spirit of the present application, such designs should fall within the scope of protection of this application. Furthermore, the word "comprising" does not exclude other elements or steps, and the word "a" preceding an element does not exclude the inclusion of "a plurality" of that element. Terms such as "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. An obstacle detection method based on multi-millimeter-wave radar data, characterized in that, include: S1, acquires multiple millimeter-wave radar data in parallel to obtain a radar data stream containing raw echo data; S2, perform FFT transformation on each radar data source to extract radar point cloud data, and mark the radar ID and timestamp to which each point cloud data belongs to, to obtain a multi-source point cloud dataset with identification information. S3. Based on the installation position and angle of each millimeter-wave radar in the vehicle coordinate system, perform coordinate transformation on the multi-source point cloud dataset with identification information, and transform the point cloud data in the local coordinate system of each radar to the vehicle coordinate system to obtain a spatially registered point cloud dataset. S4. Based on the radar ID and the three-dimensional coordinates of the point cloud in the vehicle coordinate system, identify the point cloud data located in the overlapping detection area of ​​adjacent radars, and perform fusion processing on the redundant point cloud data generated by multiple radars repeatedly detecting the same obstacle to obtain the deredundant fused point cloud data. S5. Based on the timestamp, a data association algorithm is used to match the fused point cloud data at consecutive times to establish the obstacle target trajectory. A filtering algorithm is then used to predict and update the motion state of the obstacle, resulting in a dynamic target tracking result that includes the obstacle's position and velocity. S6. Based on the redundancy-free fused point cloud data and dynamic target tracking results, obstacle information is mapped onto a preset grid map. The occupancy probability of each grid is calculated through probability update rules to obtain a real-time updated 360° obstacle grid map.

2. The obstacle detection method based on multi-millimeter-wave radar data according to claim 1, characterized in that, include: S1 acquires multiple millimeter-wave radar data in parallel, including: By deploying four millimeter-wave angle radars at the front, rear, left, and right of the vehicle, and four millimeter-wave angle radars at the front left, front right, rear left, and rear right, eight channels of millimeter-wave radar data are collected simultaneously. Among them, the detection areas of adjacent millimeter-wave angle radars are set with an overlap angle of 15° to 30°, so that the detection range of the eight millimeter-wave radars can jointly cover the 360° range around the vehicle. Millimeter-wave angle radar uses the 77GHz or 79GHz frequency band.

3. The obstacle detection method based on multi-millimeter-wave radar data according to claim 2, characterized in that, include: S2, obtains a multi-source point cloud dataset with identification information, including: Adaptive threshold detection is performed on millimeter-wave radar data to obtain denoised radar data; The denoised radar data is subjected to FFT transformation to obtain the amplitude and phase of N frequency components, where N is the number of FFT points; According to radar equations The frequency components are converted into corresponding distance information, where f is the frequency, v is the electromagnetic wave velocity, and r is the target distance. Using wavelength as the reference wavelength, distance distribution data of obstacle targets is obtained; Based on millimeter-wave angle radar, multiple built-in receiving antennas are used to acquire multi-channel received signals of the same target. For distance points with amplitudes exceeding a preset threshold, the phase information of the distance point in each receiving antenna channel is extracted, and the phase difference between adjacent antenna channels is calculated. and according to Calculate the target azimuth angle θ, where d is the distance between adjacent antennas; The distance r, azimuth angle θ, and amplitude A of each obstacle target are combined to form point cloud data. ; Add the radar ID number that generated the data and the timestamp 't' when the data was collected to each point cloud data point, resulting in the format: A multi-source point cloud dataset with identification information.

4. The obstacle detection method based on multi-millimeter-wave radar data according to claim 3, characterized in that, include: Adaptive threshold detection is performed on millimeter-wave radar data to obtain denoised radar data, including: Calculate the mean amplitude μ and standard deviation σ of the echo data in the current frame; The detection threshold is set to μ+kσ, where k is a constant from 3 to 5; Echo data with amplitudes below the detection threshold are set to zero to obtain denoised radar data, thus filtering out weak reflection clutter generated by downhole water mist and dust.

5. The obstacle detection method based on multi-millimeter-wave radar data according to claim 3, characterized in that, include: S3 yields the spatially registered point cloud dataset, including: Obtain the installation parameters of each millimeter-wave radar in the vehicle coordinate system, including: installation position coordinates. and installation angle ,in, This is the translation vector of the radar relative to the origin of the vehicle coordinate system. These are the rotation angles of the radar around the X, Y, and Z axes of the vehicle's coordinate system, respectively. Based on the radar ID of each point cloud data in the multi-source point cloud dataset with identification information, find the installation parameters of the corresponding radar. Point cloud data polar coordinates in Convert to Cartesian coordinates in the radar local coordinate system, where, , , ; According to the installation angle Calculate the rotation matrix R from the radar local coordinate system to the vehicle coordinate system; Through coordinate transformation formula The point cloud coordinates in the radar local coordinate system are transformed to the vehicle coordinate system; Retaining the point cloud amplitude A, radar ID, and timestamp t, the following format is obtained in the vehicle coordinate system: Spatial registration point cloud dataset.

6. The obstacle detection method based on multi-millimeter-wave radar data according to claim 5, characterized in that, include: S4, obtain the redundancy-free fused point cloud data, including: The overlap angle of the detection areas of adjacent millimeter-wave angular radars is used to determine the boundary of the overlapping detection area of ​​each adjacent radar pair. Based on each point cloud data in the spatially registered point cloud dataset Using coordinates Determine whether the corresponding point cloud is located within the overlapping detection area; For point cloud data located within the overlapping detection area, calculate the distance between point clouds with different radar IDs. ; Distance Less than the preset threshold The point cloud is identified as repeated detection points for the same obstacle, among which, Based on the range resolution of the millimeter-wave radar; Multiple duplicate detection points identified as the same obstacle are merged; The maximum amplitude value and earliest timestamp of the fused point cloud are retained, and duplicate point clouds are deleted to obtain the redundancy-free fused point cloud data.

7. The obstacle detection method based on multi-millimeter-wave radar data according to any one of claims 2 to 6, characterized in that, include: S5 yields dynamic target tracking results containing obstacle position and velocity, including: Based on the timestamp t in the deduplicated point cloud data, the fused point cloud data is divided into consecutive data frames in chronological order; For each point cloud in the current frame, based on the corresponding coordinates Search for candidate matching points in the previous frame of point cloud data whose distance is less than the association threshold. The association threshold is calculated based on the time interval between adjacent frames and the maximum driving speed of the vehicle. The nearest neighbor data association algorithm is used to associate the current frame point cloud with the nearest point among the candidate matching points of the previous frame to obtain the associated point cloud sequence, so as to establish the correspondence between cross-frame point clouds; Based on the successfully associated point cloud sequences, construct obstacle target trajectories. Each trajectory includes: obstacle target ID, historical location sequence, and timestamp sequence. The Kalman filter algorithm is used to estimate the state of each obstacle target trajectory, where the state vector includes the obstacle target's position (x, y, z) and velocity. The prediction step estimates the current position based on the state at the previous time step, and the update step uses the current observations to correct the prediction results. Output the ID, position (x, y, z), and velocity of each tracked obstacle target. This yields dynamic target tracking results that include the position and velocity of obstacles.

8. The obstacle detection method based on multi-millimeter-wave radar data according to claim 7, characterized in that, include: The dynamic target tracking results, which include the position and velocity of obstacles, also include: For target trajectories that have not been updated for multiple consecutive frames, determine that the target has disappeared and delete the corresponding trajectory.

9. The obstacle detection method based on multi-millimeter-wave radar data according to claim 7, characterized in that, include: S6 provides a real-time updated 360° obstacle grid map, including: Create a raster map centered on the origin of the vehicle coordinate system, and set the raster resolution so that the map covers a 360° area around the vehicle. The coordinates of each point in the deduplicated point cloud data. Mapped to the corresponding raster index in the raster map; Calculate the grid occupancy probability increment based on the point cloud data mapped to each grid. The probability update rule is adopted, which combines the historical occupancy probability of the grid with the current observation data to update the occupancy probability of each grid. Based on the occupancy probability and a preset threshold, the grid is marked as an obstacle grid, a free grid, or an unknown grid; Output a 360° obstacle grid map that includes grid occupancy status and occupancy probability.

10. An obstacle detection device based on multi-millimeter-wave radar data, characterized in that, include: The data acquisition module acquires multiple millimeter-wave radar data in parallel. The point cloud extraction module performs FFT transformation on each radar data stream to extract radar point cloud data, and marks the radar ID and timestamp to which each point cloud data belongs, resulting in a multi-source point cloud dataset with identification information. The coordinate transformation module performs coordinate transformation on the multi-source point cloud dataset with identification information according to the installation position and angle of each millimeter-wave radar in the vehicle coordinate system, transforming the point cloud data in the local coordinate system of each radar to the vehicle coordinate system, and obtaining a spatially registered point cloud dataset. The data fusion module identifies point cloud data located in the overlapping detection area of ​​adjacent radars based on radar ID and the three-dimensional coordinates of point cloud in the vehicle coordinate system. It then fuses redundant point cloud data generated by multiple radars repeatedly detecting the same obstacle to obtain redundancy-free fused point cloud data. The target tracking module uses a data association algorithm to match the fused point cloud data at consecutive times based on the timestamp, establishes the obstacle target trajectory, and uses a filtering algorithm to predict and update the motion state of the obstacle, thus obtaining a dynamic target tracking result that includes the obstacle's position and velocity. The grid map module maps obstacle information onto a preset grid map based on the deduplicated point cloud data and dynamic target tracking results. It calculates the occupancy probability of each grid cell using probability update rules to obtain a real-time updated 360° obstacle grid map.

Images