A method and system for detecting road traffic safety hazards based on 4D millimeter-wave radar

Abstract

This invention discloses a method and system for detecting road traffic safety hazards based on 4D millimeter-wave radar, comprising: S1. acquiring continuous frame point cloud sequences using 4D millimeter-wave radar at a fixed frame rate; S2. constructing a three-dimensional voxel grid through spatial discretization, performing voxel spatial modeling on the continuous multi-frame point clouds, and extracting the suddenly added voxel regions in the current frame; S3. performing point clustering and target trajectory reconstruction on the point cloud of the suddenly added voxel regions; S4. calculating the average velocity of the trajectory time window segments, and determining the behavior state transition according to the abnormal state judgment conditions; S5. outputting high-confidence abnormal state detection results through an abnormal confidence scoring function. This invention effectively detects suddenly appearing stationary targets, and improves detection accuracy and robustness by combining spatial residual analysis and temporal behavior modeling; it is easy to deploy in actual traffic monitoring systems and has good engineering application prospects.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and more specifically to a method and system for detecting road traffic safety hazards based on 4D millimeter-wave radar. Background Technology

[0002] With the development of intelligent transportation systems, 4D millimeter-wave radar, as a sensor with all-weather capability and strong anti-interference ability, is gradually being applied to scenarios such as road safety hazard monitoring, tunnel entrance sensing, and slope landslide early warning. Among these applications, identifying sudden stationary objects on the road (such as landslides, falling rocks, fallen foreign objects, and abnormally parked vehicles) through 4D millimeter-wave radar point cloud has become a key requirement in traffic safety management.

[0003] However, existing testing methods have the following limitations:

[0004] Poor ability to identify stationary objects: Traditional radar moving target detection relies heavily on Doppler velocity or inter-frame trajectory consistency, which is not effective in identifying targets that suddenly stop or have a velocity close to zero.

[0005] Unable to distinguish between background and newly added objects: In complex road structures (such as tunnel entrances, slopes, and ramps), a large number of fixed structures (such as guardrails, walls, and lampposts) will form long-term stable point cloud reflections, making it difficult to distinguish "newly appearing dangerous objects" from them;

[0006] Lack of effective behavior pattern recognition: Dangerous targets such as falling rocks, tires, and abnormal parking often have spatiotemporal behavior characteristics of "movement before stillness", and existing systems lack the ability to characterize this state transition pattern.

[0007] Therefore, how to provide a novel detection method that combines spatial change analysis and temporal behavior modeling capabilities to achieve high-precision identification and risk screening of both suddenly appearing and stationary targets is a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] In view of this, the present invention provides a method and system for detecting road traffic safety hazards based on 4D millimeter-wave radar. It aims to accurately identify newly added stationary obstacles from 4D millimeter-wave radar point clouds in complex road environments, and to use trajectory modeling to distinguish between "real abnormal targets" and "slowly moving objects / static backgrounds", thereby providing higher robustness and real-time support for intelligent traffic perception systems.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] A method for detecting road traffic safety hazards based on 4D millimeter-wave radar includes the following steps:

[0011] S1. Use 4D millimeter-wave radar to acquire continuous frame point cloud sequences at a fixed frame rate;

[0012] S2. Construct a three-dimensional voxel mesh through spatial discretization, perform voxel spatial modeling on point clouds of multiple consecutive frames, and extract the burst-new voxel regions in the current frame;

[0013] S3. Perform point clustering and target trajectory reconstruction on the point cloud of suddenly newly added voxel regions;

[0014] S4. Calculate the average velocity of the trajectory window segments and determine the behavior state transition based on the abnormal state judgment conditions;

[0015] S5. Output high-confidence abnormal state detection results through the abnormal confidence scoring function.

[0016] Preferably, the specific content of step S2 is as follows:

[0017] S21. Divide the entire three-dimensional detection space into sections with side lengths of... voxel grid A single spatial voxel has a side length of A cube, the three-dimensional detection space is uniformly divided into multiple small cubes;

[0018] S22. Use an indicator function to count the number of points in each voxel in the current frame;

[0019] S23. Calculate the historical mean of voxels in the historical frame window and the voxel residual of the current frame, and extract the bursty newly added voxel region based on the voxel residual threshold.

[0020] Preferably, the newly added voxel regions are:

[0021]

[0022] in, For the divided voxel mesh, is the voxel residual threshold, and i, j, k are the 3D indices of the voxel mesh;

[0023] The voxel residual of the current frame is:

[0024]

[0025] Select history frame window The historical mean of voxels is:

[0026]

[0027] In the In a frame, the number of points in each voxel is counted as follows:

[0028]

[0029]

[0030] in, For indicator functions, For the first Frame point cloud collection The Middle The three-dimensional coordinates of each point; For the first The number of points in a frame.

[0031] Preferably, the specific content of step S3 is as follows:

[0032] S31. Perform spatial clustering on the point cloud within the suddenly newly added voxel region to obtain a set of point clusters;

[0033] S32. Perform multi-frame tracking on each point cluster to construct a trajectory centroid sequence;

[0034] S33. Calculate the velocity vector sequence based on the trajectory centroid sequence and radar sampling interval of point clusters.

[0035] Preferred, point cluster The trajectory centroid sequence is as follows:

[0036]

[0037] in, For the first In-frame and target The set of matching points The number of points in the set. For the target center trajectory, To address the sudden emergence of new voxel regions Ω new The set of point clusters obtained by spatial clustering of the point cloud within the area. The length of the time window for target trajectory tracking;

[0038] The velocity vector sequence is:

[0039]

[0040] in, This refers to the radar sampling interval.

[0041] Preferably, the specific content of step S4 is as follows:

[0042] S41. Divide the trajectory window into an initial motion phase and a later stationary phase.

[0043] S42. Calculate the average speed within each segment;

[0044] S43. Preset abnormal state judgment conditions based on the lower limit of speed during sudden movement and the upper limit of speed during the stationary phase. If the conditions are met, it indicates that the target has come to a standstill after moving and belongs to the dangerous category.

[0045] Preferably, the initial movement stage is The average speed during the initial motion phase is:

[0046]

[0047] in, ;

[0048] The later static phase is The average velocity during the later stationary phase is:

[0049]

[0050] in, The duration of the later static phase. for Time of the first The absolute value of the velocity corresponding to each point cluster target;

[0051] The conditions for determining abnormal states are:

[0052] , ;

[0053] in, The lower limit of the speed of sudden movement, This represents the upper limit of speed during the stationary phase.

[0054] A road traffic safety hazard detection system based on 4D millimeter-wave radar, and a road traffic safety hazard detection method based on 4D millimeter-wave radar, comprising: an acquisition module, a voxel residual modeling module, a point cluster trajectory modeling module, a behavior state transition recognition module, and a result output module;

[0055] The acquisition module is used to acquire continuous frame point cloud sequences at a fixed frame rate using 4D millimeter-wave radar;

[0056] The voxel residual modeling module is used to construct a three-dimensional voxel mesh through spatial discretization, perform voxel spatial modeling on point clouds of multiple consecutive frames, and extract burst-new voxel regions in the current frame.

[0057] The point cluster trajectory modeling module is used to perform point cluster clustering and target trajectory reconstruction on point clouds of suddenly newly added voxel regions;

[0058] The behavior state transition recognition module is used to calculate the average velocity of the trajectory time window segments and to determine the behavior state transition based on the abnormal state judgment conditions.

[0059] The results output module is used to output high-confidence abnormal state detection results through the abnormal confidence scoring function.

[0060] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the aforementioned method for detecting road traffic safety hazards based on 4D millimeter-wave radar.

[0061] A processing terminal includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the aforementioned method for detecting road traffic safety hazards based on 4D millimeter-wave radar.

[0062] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a road traffic safety hazard detection method and system based on 4D millimeter-wave radar, which can effectively detect suddenly appearing stationary targets, and is particularly suitable for identifying high-risk events such as landslides, falling objects, and abnormal parking; by combining spatial residual analysis and temporal behavior modeling, the detection accuracy and robustness are improved; the algorithm structure is simple and easy to deploy in actual traffic monitoring systems, and has good engineering application prospects.

[0063] Specifically:

[0064] To address the difficulty in distinguishing stationary targets from complex background structures, a new point cluster region extraction method based on multi-frame voxel residuals is used to effectively detect newly appearing objects. To address the lack of velocity features for sudden targets, a multi-frame trajectory reconstruction mechanism is designed, using centroid velocity sequence modeling to achieve state transition recognition. To address the potential for misjudgment caused by slowly moving objects, a piecewise velocity mean and anomaly scoring function are introduced to effectively improve the discrimination ability and confidence expression of stationary abnormal targets. Combining spatial variation and temporal behavior dual-mode modeling, a low-complexity, deployable target recognition process is constructed to meet the practical application needs of millimeter-wave point cloud resolution limitations. Attached Figure Description

[0065] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0066] Figure 1 A schematic diagram of a road traffic safety hazard detection method based on 4D millimeter-wave radar provided by the present invention;

[0067] Figure 2This is a schematic diagram of a point cloud obtained from a single frame of millimeter-wave radar provided by the present invention.

[0068] Figure 3 A schematic diagram of a single spatial voxel provided by the present invention;

[0069] Figure 4 A schematic diagram illustrating the division of the entire detection space provided for this invention;

[0070] Figure 5 This is a schematic diagram of the indicator function provided by the present invention. Detailed Implementation

[0071] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0072] This invention discloses a method for detecting road traffic safety hazards based on 4D millimeter-wave radar, such as... Figure 1 As shown, it includes the following steps:

[0073] S1. Use 4D millimeter-wave radar to acquire continuous frame point cloud sequences at a fixed frame rate;

[0074] S2. Construct a three-dimensional voxel mesh through spatial discretization, perform voxel spatial modeling on point clouds of multiple consecutive frames, and extract the burst-new voxel regions in the current frame;

[0075] S3. Perform point clustering and target trajectory reconstruction on the point cloud of suddenly newly added voxel regions;

[0076] S4. Calculate the average velocity of the trajectory window segments and determine the behavior state transition based on the abnormal state judgment conditions;

[0077] S5. Output high-confidence abnormal state detection results through the abnormal confidence scoring function.

[0078] To further implement the above technical solution, the specific content of step S2 is as follows:

[0079] S21. Divide the entire three-dimensional detection space into sections with side lengths of... voxel grid A single spatial voxel has a side length of A cube, such as Figure 3 As shown, the three-dimensional detection space is uniformly divided into multiple small cubes, such as... Figure 4 As shown;

[0080] S22. Use an indicator function to count the number of points in each voxel in the current frame;

[0081] S23. Calculate the historical mean of voxels in the historical frame window and the voxel residual of the current frame, and extract the bursty newly added voxel region based on the voxel residual threshold.

[0082] In this embodiment, a continuous frame point cloud sequence is acquired by sampling at a fixed frame rate using a 4D millimeter-wave radar. The point cloud acquired in one frame is as follows Figure 2 As shown.

[0083] To further implement the above technical solution, the newly added voxel region is as follows:

[0084]

[0085] in, For the divided voxel mesh, The threshold for voxel residual scoring;

[0086] In this embodiment,

[0087]

[0088] Where i, j, and k are the three-dimensional indices of the voxel mesh, corresponding to the x, y, and z axes in three-dimensional space, respectively;

[0089] The voxel residual of the current frame is:

[0090]

[0091] Select history frame window The historical mean of voxels is:

[0092]

[0093] In the In a frame, the number of points in each voxel is counted as follows:

[0094]

[0095]

[0096] in, For indicator functions, such as Figure 5 , For the first Frame point cloud collection The Middle The three-dimensional coordinates of each point; For the first The number of points in a frame.

[0097] To further implement the above technical solution, the specific content of step S3 is as follows:

[0098] S31. Perform spatial clustering on the point cloud within the suddenly newly added voxel region to obtain a set of point clusters;

[0099] S32. Perform multi-frame tracking on each point cluster to construct a trajectory centroid sequence;

[0100] S33. Calculate the velocity vector sequence based on the trajectory centroid sequence and radar sampling interval of point clusters.

[0101] To further implement the above technical solution, point clusters The trajectory centroid sequence is as follows:

[0102]

[0103] in, For the first In-frame and target The set of matching points The number of points in the set. For the target center trajectory, To address the sudden emergence of new voxel regions Ω new The set of point clusters obtained by spatial clustering of the point cloud within the area. The length of the time window for target trajectory tracking;

[0104] The velocity vector sequence is:

[0105]

[0106] in, This refers to the radar sampling interval.

[0107] To further implement the above technical solution, the specific content of step S4 is as follows:

[0108] S41. Divide the trajectory window into an initial motion phase and a later stationary phase.

[0109] S42. Calculate the average speed within each segment;

[0110] S43. Preset abnormal state judgment conditions based on the lower limit of speed during sudden movement and the upper limit of speed during the stationary phase. If the conditions are met, it indicates that the target has come to a standstill after moving and belongs to the dangerous category, such as rolling stones, falling tires, cars that suddenly stop, etc.

[0111] To further implement the above technical solution, the total duration will be tracked. Divided into the initial movement stage and the later static phase The initial exercise phase lasted for [duration]. The duration of the later static phase is ;

[0112] The initial stage of movement is The average speed during the initial motion phase is:

[0113]

[0114] in, ;

[0115] The later static phase is The average velocity during the later stationary phase is:

[0116]

[0117] in, The duration of the later static phase. for Time of the first The absolute value of the velocity corresponding to each point cluster target;

[0118] The conditions for determining abnormal states are:

[0119] , ;

[0120] in, The lower limit of the speed of sudden movement, This represents the upper limit of speed during the stationary phase.

[0121] In this embodiment, the anomaly confidence scoring function is:

[0122]

[0123] in, These are weighting coefficients. The larger the value, the stronger the abnormality.

[0124] A road traffic safety hazard detection system based on 4D millimeter-wave radar and a road traffic safety hazard detection method based on 4D millimeter-wave radar include: an acquisition module, a voxel residual modeling module, a point cluster trajectory modeling module, a behavior state transition recognition module, and a result output module.

[0125] The acquisition module is used to acquire continuous frame point cloud sequences at a fixed frame rate using 4D millimeter-wave radar;

[0126] The voxel residual modeling module is used to construct a three-dimensional voxel mesh through spatial discretization, perform voxel spatial modeling on point clouds of multiple consecutive frames, and extract burst-new voxel regions in the current frame.

[0127] The point cluster trajectory modeling module is used to perform point cluster clustering and target trajectory reconstruction on point clouds of suddenly newly added voxel regions;

[0128] The behavior state transition recognition module is used to calculate the average velocity of the trajectory time window segments and to determine the behavior state transition based on the abnormal state judgment conditions.

[0129] The results output module is used to output high-confidence abnormal state detection results through the abnormal confidence scoring function.

[0130] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for detecting road traffic safety hazards based on 4D millimeter-wave radar.

[0131] A processing terminal includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements a method for detecting road traffic safety hazards based on 4D millimeter-wave radar.

[0132] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0133] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting road traffic safety hazards based on 4D millimeter-wave radar, characterized in that, Includes the following steps: S1. Use 4D millimeter-wave radar to acquire continuous frame point cloud sequences at a fixed frame rate; S2. Construct a three-dimensional voxel mesh through spatial discretization, perform voxel spatial modeling on point clouds of multiple consecutive frames, and extract the burst-new voxel regions in the current frame; S3. Perform point clustering and target trajectory reconstruction on the point cloud of suddenly newly added voxel regions; S4. Calculate the average velocity of the trajectory window segments and determine the behavior state transition based on the abnormal state judgment conditions; S5. Output high-confidence abnormal state detection results through the abnormal confidence scoring function; The specific content of step S2 is as follows: S21. Divide the entire three-dimensional detection space into sections with side lengths of... voxel grid A single spatial voxel has a side length of A cube, the three-dimensional detection space is uniformly divided into multiple small cubes; S22. Use an indicator function to count the number of points in each voxel in the current frame; S23. Calculate the historical mean of voxels in the historical frame window and the voxel residual of the current frame, and extract the bursty newly added voxel region based on the voxel residual threshold; The specific content of step S4 is as follows: S41. Divide the trajectory window into an initial motion phase and a later stationary phase. S42. Calculate the average speed within each segment; S43. Preset abnormal state judgment conditions based on the lower limit of speed during sudden movement and the upper limit of speed during the stationary phase. If the conditions are met, it indicates that the target has come to a standstill after moving and belongs to the dangerous category.

2. The method for detecting road traffic safety hazards based on 4D millimeter-wave radar according to claim 1, characterized in that, The newly added voxel regions are: ； in, For the divided voxel mesh, is the voxel residual threshold, and i, j, k are the 3D indices of the voxel mesh; The voxel residual of the current frame is: ； Select history frame window The historical mean of voxels is: ； In the In a frame, the number of points in each voxel is counted as follows: ； ； in, For indicator functions, For the first Frame point cloud collection The Middle The three-dimensional coordinates of each point; For the first The number of points in a frame.

3. The method for detecting road traffic safety hazards based on 4D millimeter-wave radar according to claim 1, characterized in that, The specific content of step S3 is as follows: S31. Perform spatial clustering on the point cloud within the suddenly newly added voxel region to obtain a set of point clusters; S32. Perform multi-frame tracking on each point cluster to construct a trajectory centroid sequence; S33. Calculate the velocity vector sequence based on the trajectory centroid sequence and radar sampling interval of point clusters.

4. The method for detecting road traffic safety hazards based on 4D millimeter-wave radar according to claim 3, characterized in that, Point clusters The trajectory centroid sequence is as follows: ； in, For the first In-frame and target The set of matching points The number of points in the set. For the target center trajectory, To address the sudden emergence of new voxel regions Ω new The set of point clusters obtained by spatial clustering of the interior point cloud. The length of the time window for target trajectory tracking; The velocity vector sequence is: ； in, This refers to the radar sampling interval.

5. The method for detecting road traffic safety hazards based on 4D millimeter-wave radar according to claim 1, characterized in that, The initial stage of movement is The average speed during the initial motion phase is: ； in, , The length of the time window for target trajectory tracking; The later static phase is The average velocity during the later stationary phase is: ； in, The duration of the later static phase. for Time of the first The absolute value of the velocity corresponding to each point cluster target; The conditions for determining abnormal states are: ； ； in, The lower limit of the speed of sudden movement, This represents the upper limit of speed during the stationary phase.

6. A road traffic safety hazard detection system based on 4D millimeter-wave radar, characterized in that, A road traffic safety hazard detection method based on 4D millimeter-wave radar according to any one of claims 1-5 includes: an acquisition module, a voxel residual modeling module, a point cluster trajectory modeling module, a behavior state transition recognition module, and a result output module. The acquisition module is used to acquire continuous frame point cloud sequences at a fixed frame rate using 4D millimeter-wave radar; The voxel residual modeling module is used to construct a three-dimensional voxel mesh through spatial discretization, perform voxel spatial modeling on point clouds of multiple consecutive frames, and extract burst-new voxel regions in the current frame. The point cluster trajectory modeling module is used to perform point cluster clustering and target trajectory reconstruction on point clouds of suddenly newly added voxel regions; The behavior state transition recognition module is used to calculate the average velocity of the trajectory time window segments and to determine the behavior state transition based on the abnormal state judgment conditions. The results output module is used to output high-confidence abnormal state detection results through the abnormal confidence scoring function.

7. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements a road traffic safety hazard detection method based on 4D millimeter-wave radar as described in any one of claims 1-5.

8. A processing terminal, characterized in that, It includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the computer program, it implements the road traffic safety hazard detection method based on 4D millimeter-wave radar as described in any one of claims 1-5.

Images