Road traffic potential safety hazard detection method and system based on 4D millimeter wave radar

Abstract

The invention discloses a road traffic potential safety hazard detection method and system based on a 4D millimeter wave radar. The method comprises the following steps: S1, collecting a continuous frame point cloud sequence at a fixed frame rate by using the 4D millimeter wave radar; s2, constructing a three-dimensional voxel grid through spatial discretization, carrying out voxel spatial modeling on continuous multi-frame point clouds, and extracting a sudden newly-added voxel region of a current frame; s3, performing point cluster clustering and target trajectory reconstruction on the point cloud of the suddenly added voxel region; s4, calculating a trajectory time window segmentation speed mean value, and carrying out behavior state transition judgment according to an abnormal state judgment condition; s5, outputting an abnormal state detection result with high confidence through an abnormal confidence scoring function; according to the method, the sudden static target is effectively detected, and the detection precision and robustness are improved by combining spatial residual analysis and time behavior modeling; the method is easy to deploy in an actual traffic monitoring system and has a good engineering application prospect.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, and more particularly to a road traffic safety hazard detection method and system based on 4D millimeter wave radar. Background Art

[0002] With the development of intelligent transportation systems, 4D millimeter-wave radar, as an all-weather sensor with strong anti-interference capabilities, has gradually been applied to scenarios such as road safety hazard monitoring, tunnel entrance perception, and slope landslide warning. Among them, the identification of sudden stationary objects on the road (such as landslides, fallen foreign objects, and abnormally parked vehicles) through 4D millimeter-wave radar point clouds has become a key requirement in traffic safety management.

[0003] However, existing detection solutions have the following main limitations:

[0004] Poor ability to identify stationary objects: Traditional radar motion target detection relies heavily on Doppler velocity or inter-frame trajectory consistency, and is ineffective at identifying targets that suddenly stop or have a speed close to zero.

[0005] Inability to distinguish between background and newly added objects: In complex road structures (such as tunnel entrances, slopes, and ramps), a large number of fixed structures (such as guardrails, walls, and light poles) will form long-term stable point cloud reflections, making it difficult to distinguish "newly appeared dangerous objects" from them;

[0006] Lack of effective behavioral pattern recognition: Dangerous targets such as falling rocks, tires, and abnormal parking often have the spatiotemporal behavioral characteristics of "first moving and then stationary". The existing system lacks the ability to characterize this state transition pattern.

[0007] Therefore, how to provide a novel detection method that combines spatial change analysis and temporal behavior modeling capabilities to achieve high-precision identification and risk screening of sudden-appearing and stationary targets is a technical problem that technical personnel in this field urgently need to solve. Summary of the Invention

[0008] In view of this, the present invention provides a road traffic safety hazard detection method and system based on 4D millimeter-wave radar, aiming to accurately identify new static obstacles from the 4D millimeter-wave radar point cloud in complex road environments, and use trajectory modeling to distinguish "real abnormal targets" from "slow-moving objects / static background", providing higher robustness and real-time support for intelligent traffic perception systems.

[0009] In order to achieve the above object, the present invention adopts the following technical solutions:

[0010] A road traffic safety hazard detection method based on 4D millimeter wave radar includes the following steps:

[0011] S1. Collecting continuous frame point cloud sequence with 4D millimeter wave radar at fixed frame rate;

[0012] S2. Constructing three-dimensional voxel grid through spatial discretization, voxel space modeling for continuous multi-frame point cloud, and extracting current frame burst new voxel region;

[0013] S3. Point cluster clustering and target trajectory reconstruction for point cloud of burst new voxel region;

[0014] S4. Calculating trajectory time window segmented speed average, and performing behavior state transition judgment according to abnormal state judgment condition;

[0015] S5. Outputting high confidence abnormal state detection result through abnormal confidence score function.

[0016] Preferably, the specific content of step S2 is:

[0017] S21. Dividing the entire three-dimensional detection space into voxel grid with side length of , single space voxel is a cube with side length of , and the three-dimensional detection space is uniformly divided into multiple small cubes;

[0018] S22. Using indicator function to count the number of points in each voxel of the current frame;

[0019] S23. Calculating voxel historical average of history frame window and voxel residual of current frame, and extracting burst new voxel region according to voxel residual threshold.

[0020] Preferably, the burst new voxel region is:

[0021] Wherein, is the divided voxel grid, is the voxel residual threshold, and i, j, k are three-dimensional indexes of the voxel grid;

[0022] The current frame voxel residual is:

[0023] Selecting history frame window , the voxel historical average is:

[0024] In the first frame, the number of points in each voxel is:

[0025]

[0026] ​wherein, is an indicator function, is the frame point cloud set is the three-dimensional coordinate of the i-th point in the frame; is the number of points in the frame.

[0027] Preferably, the specific content of step S3 is:

[0028] S31. Spatially clustering the point cloud in the burst-added voxel region to obtain a point cluster set;

[0029] S32. Multi-frame tracking each point cluster to construct a trajectory centroid sequence;

[0030] S33. Calculating a velocity vector sequence based on the trajectory centroid sequence of the point cluster and the radar sampling interval.

[0031] Preferably, the trajectory centroid sequence of the point cluster is:

[0032] wherein, is the point set matched with the target in the i-th frame, is the number of points in the set, is the target center trajectory, is the point cluster set obtained by spatially clustering the point cloud in the burst-added voxel region Ω new is the time window length of the target trajectory tracking; The velocity vector sequence is:

[0033]

[0034] wherein, is the radar sampling interval.

[0035] Preferably, the specific content of step S4 is:

[0036] S41. Splitting the trajectory window into an initial motion stage and a later stationary stage,

[0037] S42. Calculating the average speed in each segment, respectively;

[0038] S43. According to the abnormal state judgment condition preset by the lower limit of the burst motion speed and the upper limit of the stationary stage speed, if the condition is met, it indicates that the target is stationary after moving, which belongs to the dangerous category.

[0039] Preferably, the initial motion stage is ​​, the average speed of the initial motion stage is:

[0040] wherein, ;

[0041] the late stationary stage is , the average speed of the late stationary stage is:

[0042] wherein, is the duration of the late stationary stage, is the time point corresponding to the absolute value of the speed of the point cluster target;

[0043] The abnormal state determination condition is: , ;

[0044] wherein, is the lower limit of the speed of the sudden motion, is the upper limit of the speed of the stationary stage.

[0045] A road traffic safety hazard detection system based on a 4D millimeter wave radar, based on the road traffic safety hazard detection method based on the 4D millimeter wave radar, comprising: a collection module, a voxel residual modeling module, a point cluster trajectory modeling module, a behavior state transition identification module and a result output module;

[0046] The collection module is used for collecting continuous frame point cloud sequences at a fixed frame rate by using a 4D millimeter wave radar.

[0047] The voxel residual modeling module is used for constructing a three-dimensional voxel grid through spatial discretization, voxel space modeling of continuous multiple frames of point clouds, and extraction of current frame sudden new voxel regions.

[0048] The point cluster trajectory modeling module is used for point cluster clustering and target trajectory reconstruction of the point cloud of the sudden new voxel region.

[0049] The behavior state transition identification module is used for calculating the average speed of the trajectory time window segmentation, and determining the behavior state transition according to the abnormal state determination condition.

[0050] The result output module is used for outputting the high-confidence abnormal state detection result through an abnormal confidence scoring function.

[0051] A computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement the road traffic safety hazard detection method based on the 4D millimeter wave radar.

[0052] A processing terminal comprising a memory and a processor, the memory storing a computer program capable of running on the processor, and the processor implements the method for detecting road traffic safety hazards based on 4D millimeter wave radar when executing the computer program.

[0053] According to the above technical solution, compared with the prior art, the present application provides a method and system for detecting road traffic safety hazards based on 4D millimeter wave radar, which can effectively detect sudden static targets, and is particularly suitable for identifying high-risk events such as landslides, falling objects, and abnormal parking; by combining spatial residual analysis and time behavior modeling, the detection accuracy and robustness are improved; the algorithm structure is simple and easy to deploy in actual traffic monitoring systems, and has good engineering application prospect;

[0054] Specifically:

[0055] In view of the problem that static targets and complex background structures are difficult to distinguish, a new point cluster region extraction method based on multi-frame voxel residual is used to effectively detect new objects; in view of the problem that the sudden target lacks speed features, a target multi-frame trajectory reconstruction mechanism is designed to realize state transition identification through centroid speed sequence modeling; in view of the problem that slow-moving objects may cause misjudgment, a segmented speed mean and abnormal score function is introduced to effectively improve the discrimination ability and confidence expression of static abnormal targets; by combining spatial variation and time behavior dual modeling, a low-complexity and deployable target identification process is constructed to adapt to the actual application requirements of limited millimeter wave point cloud resolution. BRIEF DESCRIPTION OF DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0057] Figure 1 A method for detecting road traffic safety hazards based on 4D millimeter wave radar provided by the present application is shown in the figure; Figure 2 A point cloud diagram obtained by a millimeter wave radar provided by the present application is shown in the figure; Figure 3 A single spatial voxel diagram provided by the present application is shown in the figure; Figure 4 A division diagram of the entire detection space provided by the present application is shown in the figure; Figure 5 A function diagram provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0059] The embodiment of the present invention discloses a method for detecting road traffic safety hazards based on 4D millimeter wave radar. Figure 1 As shown, the following steps are included:

[0060] S1. Use 4D millimeter-wave radar to collect continuous frame point cloud sequences at a fixed frame rate;

[0061] S2. Construct a 3D voxel grid by spatial discretization, perform voxel space modeling on the continuous multi-frame point cloud, and extract the newly added voxel region in the current frame;

[0062] S3. Perform point clustering and target trajectory reconstruction on the point cloud of the suddenly added voxel area;

[0063] S4. Calculate the mean velocity of each segment of the trajectory window and determine the behavior state transition based on the abnormal state determination conditions;

[0064] S5. Output high-confidence abnormal state detection results through the abnormal confidence scoring function.

[0065] In order to further implement the above technical solution, the specific content of step S2 is:

[0066] S21. Divide the entire three-dimensional detection space into two parts with side lengths of Voxel grid of , a single spatial voxel has a side length of Cube, such as Figure 3 As shown, the three-dimensional detection space is uniformly divided into multiple small cubes, such as Figure 4 As shown;

[0067] S22. Count the number of points in each voxel of the current frame using the indicator function;

[0068] S23. Calculate the historical mean of the voxels in the historical frame window and the voxel residual of the current frame, and extract the sudden new voxel area according to the voxel residual threshold.

[0069] In this embodiment, a 4D millimeter wave radar is used to sample at a fixed frame rate to collect a continuous frame point cloud sequence. , the point cloud obtained in one frame is as follows Figure 2 shown.

[0070] In order to further implement the above technical solution, the sudden new voxel areas are:

[0071] in, is the divided voxel grid, is the voxel residual score threshold;

[0072] In this embodiment,

[0073]

[0074] Among them, i, j, and k are the three-dimensional indices of the voxel grid, corresponding to the x, y, and z axes of the three-dimensional space respectively;

[0075] The voxel residual of the current frame is:

[0076] Selected History Frame Window , the voxel historical mean is:

[0077] In the In a frame, the number of points in each voxel is counted as:

[0078]

[0079] in, is an indicator function, such as Figure 5 , For the Frame point cloud collection Middle The three-dimensional coordinates of a point; For the The number of points in the frame.

[0080] In order to further implement the above technical solution, the specific content of step S3 is:

[0081] S31. spatially clustering the point cloud within the suddenly added voxel region to obtain a point cluster set;

[0082] S32. Perform multi-frame tracking on each point cluster to construct a trajectory centroid sequence;

[0083] S33. Calculate the velocity vector sequence based on the trajectory centroid sequence of the point cluster and the radar sampling interval.

[0084] In order to further implement the above technical solution, point cluster The trajectory centroid sequence of is:

[0085] wherein, is the first frame, is the set of points in the frame matching the target is the number of points in the set, is the target center trajectory, is the set of point clusters obtained by spatial clustering of the point cloud in the burst added voxel region Ω new , and is the length of the time window for target trajectory tracking.

[0086] The sequence of velocity vectors is:

[0087] wherein, is the radar sampling interval.

[0088] To further implement the above technical solution, the specific content of step S4 is:

[0089] S41. Split the trajectory window into an initial motion stage and a later stationary stage,

[0090] S42. Calculate the average speed in each segment, respectively;

[0091] S43. According to the lower limit of the speed of the burst motion and the upper limit of the speed of the stationary stage, preset the abnormal state judgment condition, if the condition is met, it indicates that the target is stationary after moving, which belongs to the dangerous category, such as rolling stones, fallen tires, suddenly stopped cars, etc.

[0092] To further implement the above technical solution, the total tracking time is divided into an initial motion stage and a later stationary stage ; the initial motion stage lasts for , and the later stationary stage lasts for ;

[0093] The initial motion stage is , and the average speed of the initial motion stage is:

[0094] wherein, ;

[0095] The later stationary stage is , and the average speed of the later stationary stage is:

[0096] wherein, is the duration of the later stationary stage, is the time absolute value of the speed corresponding to the target of the point cluster;

[0097] The abnormal state determination condition is: , ;

[0098] wherein, is the lower limit of the speed of the sudden movement, is the upper limit of the speed of the stationary phase.

[0099] In this embodiment, the abnormal confidence score function is:

[0100] wherein, is a weighting coefficient, the greater the abnormality is stronger.

[0101] A road traffic safety hazard detection system based on a 4D millimeter wave radar, based on a road traffic safety hazard detection method based on a 4D millimeter wave radar, comprising: a collection module, a voxel residual modeling module, a point cluster trajectory modeling module, a behavior state transition identification module and a result output module;

[0102] The collection module is used to collect continuous frame point cloud sequences at a fixed frame rate by using a 4D millimeter wave radar.

[0103] The voxel residual modeling module is used to construct a three-dimensional voxel grid by spatial discretization, model the voxel space of the continuous multiple frames of point clouds, and extract the current frame of sudden new voxel region.

[0104] The point cluster trajectory modeling module is used to perform point cluster clustering and target trajectory reconstruction on the point cloud of the sudden new voxel region.

[0105] The behavior state transition identification module is used to calculate the trajectory time window segmented speed mean value, and determine the behavior state transition according to the abnormal state determination condition.

[0106] The result output module is used to output the high-confidence abnormal state detection result through the abnormal confidence score function.

[0107] A computer readable storage medium having a computer program stored thereon, the computer program being executed by a processor to implement a road traffic safety hazard detection method based on a 4D millimeter wave radar.

[0108] A processing terminal comprising a memory and a processor, the memory storing a computer program executable on the processor, and the processor executing the computer program to implement a road traffic safety hazard detection method based on a 4D millimeter wave radar.

[0109] The various embodiments described in this specification are presented by way of example, and each embodiment is presented with the understanding that it will not limit the scope of the disclosure. Each embodiment is provided to highlight a certain aspect of the disclosure, and the scope of the disclosure is not limited to that embodiment. The same or similar elements are denoted by the same or similar reference numbers throughout the drawings and the specification.

[0110] The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A road traffic safety hazard detection method based on 4D millimeter wave radar, characterized in that: The following steps are involved: S1. Use 4D millimeter-wave radar to collect continuous frame point cloud sequences at a fixed frame rate; S2. Construct a 3D voxel grid by spatial discretization, perform voxel space modeling on the continuous multi-frame point cloud, and extract the newly added voxel region in the current frame; S3. Perform point clustering and target trajectory reconstruction on the point cloud of the suddenly added voxel area; S4. Calculate the mean velocity of each segment of the trajectory window and determine the behavior state transition based on the abnormal state determination conditions; S5. Output high-confidence abnormal state detection results through the abnormal confidence scoring function.

2. The method for detecting road traffic safety hazards based on 4D millimeter wave radar according to claim 1, characterized in that: The specific content of step S2 is: S21. Divide the entire three-dimensional detection space into two parts with side lengths of Voxel grid of , a single spatial voxel has a side length of The three-dimensional detection space is evenly divided into multiple small cubes; S22. Count the number of points in each voxel of the current frame using the indicator function; S23. Calculate the historical mean of the voxels in the historical frame window and the voxel residual of the current frame, and extract the sudden new voxel area according to the voxel residual threshold.

3. The method for detecting road traffic safety hazards based on 4D millimeter wave radar according to claim 2, characterized in that: The areas where new voxels are added suddenly are: ； in, is the divided voxel grid, is the voxel residual threshold, i, j, k are the three-dimensional indices of the voxel grid; The voxel residual of the current frame is: ； Selected History Frame Window , the voxel historical mean is: ； In the In a frame, the number of points in each voxel is counted as: ； ； in, is the indicator function, For the Frame point cloud collection Middle The three-dimensional coordinates of a point; For the The number of points in the frame.

4. The method for detecting road traffic safety hazards based on 4D millimeter wave radar according to claim 1, characterized in that: The specific content of step S3 is: S31. spatially clustering the point cloud within the suddenly added voxel region to obtain a point cluster set; S32. Perform multi-frame tracking on each point cluster to construct a trajectory centroid sequence; S33. Calculate the velocity vector sequence based on the trajectory centroid sequence of the point cluster and the radar sampling interval.

5. The method for detecting road traffic safety hazards based on 4D millimeter wave radar according to claim 4, characterized in that: Point cluster The trajectory centroid sequence of is: ； in, For the Frame and target The set of matched points, is the number of points in the set, is the target center trajectory, To add a new voxel area Ω for the sudden new The point cluster set obtained by spatial clustering of the inner point cloud, The length of the time window for target trajectory tracking; The velocity vector sequence is: ； in, is the radar sampling interval.

6. The method for detecting road traffic safety hazards based on 4D millimeter wave radar according to claim 1, characterized in that: The specific content of step S4 is: S41. Split the trajectory window into an early motion phase and a late static phase. S42. Calculate the average speed in each section respectively; S43. Preset abnormal state determination conditions based on the lower speed limit of the sudden movement and the upper speed limit of the stationary phase. If the conditions are met, it indicates that the target is stationary after moving and belongs to the dangerous category.

7. The method for detecting road traffic safety hazards based on 4D millimeter wave radar according to claim 6, characterized in that: The initial exercise stage is , the average speed in the initial movement stage is: ； in, ; The late stationary phase is , the mean velocity in the later stationary stage is: ； in, is the duration of the late stationary phase, for Moment The absolute value of the velocity corresponding to the point cluster target; The abnormal status determination conditions are: ， ； in, is the lower limit of the speed of sudden movement, The upper speed limit of the stationary phase.

8. A road traffic safety hazard detection system based on 4D millimeter wave radar, characterized in that: A road traffic safety hazard detection method based on 4D millimeter wave radar according to any one of claims 1 to 7, comprising: an acquisition module, a voxel residual modeling module, a point cluster trajectory modeling module, a behavior state transition recognition module, and a result output module; An acquisition module is used to acquire continuous frame point cloud sequences at a fixed frame rate using a 4D millimeter-wave radar; The voxel residual modeling module is used to construct a three-dimensional voxel grid through spatial discretization, perform voxel space modeling on continuous multi-frame point clouds, and extract the newly added voxel area in the current frame; Point cluster trajectory modeling module, used to perform point clustering and target trajectory reconstruction on the point cloud of the suddenly added voxel area; The behavior state transition recognition module is used to calculate the mean velocity of the trajectory time window segment and make behavior state transition judgments based on the abnormal state judgment conditions; The result output module is used to output high-confidence abnormal state detection results through the abnormal confidence scoring function.

9. A computer-readable storage medium, characterized in that A computer program is stored thereon, which, when executed by a processor, implements a road traffic safety hazard detection method based on 4D millimeter-wave radar as described in any one of claims 1 to 7.

10. A processing terminal, characterized in that: It includes a memory and a processor, the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, it implements a road traffic safety hazard detection method based on 4D millimeter wave radar as described in any one of claims 1 to 7.

Images