Intelligent driving collision monitoring method based on trajectory prediction

Abstract

The invention discloses an intelligent driving collision monitoring method based on trajectory prediction, and the method comprises the steps: taking the following features of a vehicle, the interactive behavior features and the eye movement features of a driver as an LSTM-Attention deep learning model, inputting a predicted vehicle moving trajectory, and carrying out the prediction of the vehicle moving trajectory through employing a directed bounding box method on the basis of the predicted trajectory, and OBB) detecting whether there is a risk conflict possibility. According to the method, after the existence of the collision risk possibility is confirmed, a collision early-warning system based on TTC (Time to Collision) is adopted to perform early-warning grading on the severity of the risk collision, and corresponding early-warning measures are taken for different risk grades, so that a driver can timely obtain the potential collision risk condition to take corresponding risk avoiding measures, and the driving safety is improved. And the safety of road traffic is improved.

Description

Technical Field

[0001] The present invention belongs to the field of traffic safety, and particularly relates to an intelligent driving collision monitoring method based on trajectory prediction. Background Art

[0002] With the rapid development of vehicle networking and machine learning technologies, traffic construction is accelerating towards the intelligent direction, making it possible for vehicles to predict risks in advance and actively intervene. By real-time monitoring the running state of vehicles, predicting potential risks and actively issuing safety warnings is crucial for ensuring the life and property safety of drivers. However, traditional vehicle collision warning methods have certain limitations. One method is based on the distance and speed between the vehicle and the preceding vehicle, and realizes warning by defining the risk of the current state. However, this method ignores the reaction time of the driver. When the system triggers a dangerous state, it is often difficult for the driver to react in time, and may even make misoperations due to nervousness, resulting in new safety hazards. Another method is based on collision accident data, and evaluates the risk of the vehicle in real time by constructing a relationship model between the traffic environment, the driver's state and the severity of the accident. However, this method relies on accident data recorded afterwards, and these data usually have the disadvantages of insufficient timeliness, limited data quality and quantity, and are also easily affected by the subjective factors of the recorder. This makes it difficult for the risk assessment model based on accident data to ensure the accuracy and real-time of the warning. In addition, the existing market has insufficient research on the gaze allocation weights of drivers with different driving styles in different regions. There is an urgent need to develop a collision monitoring system that integrates and analyzes multi-module data to provide higher accuracy and human-like results.

[0003] Therefore, how to effectively combine methods such as using eye movement tracking technology to analyze the driver's attention state, using trajectory prediction algorithms to predict the vehicle's driving path, real-time monitoring of potential dangers in the surrounding environment through a collision warning system, and multi-data fusion technology to comprehensively process vehicle dynamic information and external environment data, accurately predict the risks that the vehicle may face in the future period under the current driving state, and actively provide more accurate safety warning information to the driver, so as to reserve sufficient reaction time for him, which is of great significance for improving traffic safety. Summary of the Invention

[0004] To solve the above problems, the present invention proposes an intelligent driving collision monitoring method based on trajectory prediction, which uses the collected following characteristics of the host vehicle, interaction behavior characteristics and driver's eye movement characteristics to predict the running trajectory of the target vehicle in the future period and judge the possible safety risks. The invention method has a theoretical supporting effect on single-vehicle level warning in general complex traffic scenarios.

[0005] To achieve the above object, the present invention provides the following solutions:

[0006] An intelligent driving collision monitoring method based on trajectory prediction, comprising the following steps:

[0007] Step 1: Use the following features of the host vehicle as the input of the deep learning model to predict the vehicle's running trajectory: following vehicle characteristics, interaction behavior characteristics, and driver eye movement characteristics;

[0008] Step 2: After obtaining the coordinates of each vertex of the bounding box of the vehicle under test, select one axis from the detection axes of the two vehicles as the projection axis, and detect whether the projections of the two vehicles overlap. Repeat the above steps until all four axes of the two vehicles are detected;

[0009] Step 3: Calculate the time to collision (TTC) through the predicted vehicle trajectory to judge the risk level;

[0010] Step 4: Take different levels of warning measures for different levels of risk conflicts.

[0011] A further improvement of the present invention lies in: in step 1, obtaining the future running trajectory of the vehicle, specifically including:

[0012] Using the following features of the host vehicle as the input of the deep learning model to predict the vehicle's running trajectory, and the specific data feature indicators extracted include:

[0013] The lateral and longitudinal relative distances between the vehicle at time t and the adjacent leading vehicle;

[0014] The lateral and longitudinal speeds of the vehicle at time t;

[0015] The lateral and longitudinal speeds of the adjacent leading vehicle of the vehicle at time t;

[0016] The line-of-sight angles between the driver of the vehicle at time t and the pedestrians on the left and right sides respectively;

[0017] The lateral and longitudinal relative distances between the vehicle and the interactive pedestrian on the left at time t;

[0018] The lateral and longitudinal speeds of the interactive pedestrian on the left side of the vehicle at time t;

[0019] The lateral and longitudinal relative distances between the vehicle and the interactive pedestrian on the right at time t;

[0020] The lateral and longitudinal speeds of the interactive pedestrian on the right side of the vehicle at time t;

[0021] The driver's gaze position at time t;

[0022] The driver's gaze duration at time t;

[0023] Input the above extracted features into the LSTM-Attention model to obtain more accurate predicted trajectory points.

[0024] The LSTM consists of structures such as an input gate, a forget gate, and an output gate. The "gate" structure selectively controls the flow of information and can better capture time dependencies. The present invention utilizes the "gate" structure to mine historical decision-making and state features, adaptively retain and forget historical information, and thus output a "memory vector", which can accurately reflect the decision-making process of the driver in the gradually disappearing memory effect. The Attention mechanism is used to adaptively and differentially assign weights to the spatio-temporal features of vehicle operation, enhance the model's ability to focus on key spatio-temporal information, and more sensitively capture the dynamic changes of interaction features (such as relative direction, relative distance, line-of-sight angle, etc.), thereby improving the prediction accuracy and further enhancing the accuracy of subsequent collision warning analysis.

[0025] A further improvement of the present invention lies in: in step 2, obtaining the coordinates of each vertex of the bounding box of the vehicle to be measured, which specifically includes:

[0026] Assume that the length of the bounding box of the vehicle to be measured is 5.0 m and the width is 2.0 m. Based on existing research, the traffic severity threshold and the early warning time are used to improve the bounding box parameters to achieve conflict detection and conflict severity detection. Assume that the length of the bounding box of the vehicle to be measured is L h , and the width is K h , and T re is the early warning time, and the formula is as follows:

[0027] L h = 5 + v x *T re

[0028] K h = 2 + v x *T re

[0029] According to the predicted trajectory result of step 1, taking the severe conflict threshold of 0.6 s as an example, the coordinates of each vertex of the two bounding boxes are solved according to the following formula.

[0030]

[0031] Among them, D1 - D4 are the vertices of vehicle 1; D5 - D8 are the vertices of vehicle 2; x D is the abscissa of the vertex; y D is the ordinate of the vertex; θ(t) is the direction angle of the target vehicle.

[0032] A further improvement of the present invention lies in: in step 2, detecting whether the projections of the two vehicles overlap, which specifically includes:

[0033] After obtaining the coordinates of each vertex of the vehicle, the risk of collision of the vehicle can be checked based on the separating axis principle. The oriented bounding box algorithm detects collisions based on the separating axis theorem. The separating axis theorem determines whether a collision occurs by judging whether there is an overlap in the projections of two convex polygons at any angle. However, in actual scenarios, it is obviously not in line with the high timeliness requirements of collision detection to project all angles of the convex polygon. Therefore, the oriented bounding box algorithm replaces the actual complex vehicle shape with a simple rectangle that tightly encloses the vehicle to be measured. At this time, it is only necessary to project the bounding boxes of the two vehicles to be measured at four key angles to determine whether there is an overlap to achieve collision detection. If the projections at the four detection angles all overlap, the two vehicles collide. If any one of the projections does not overlap, there is no collision.

[0034] A further improvement of the present invention lies in: in step 3, calculating the TTC parameter of the target vehicle to judge the risk level, which specifically includes:

[0035] The most likely accident type of a vehicle in a traffic scenario is vehicle collision. TTC is an important traffic safety indicator used to evaluate the possibility and time of a collision between two traffic participants. It represents how long it will take for two vehicles or a moving object and a potential collision target to collide under the current motion state. The specific formula is as follows:

[0036]

[0037] where d is the relative distance between the target vehicle and the vehicle in front; v r is the relative velocity of the target vehicle and the vehicle in front along the collision direction.

[0038] If two objects approach each other, v r > 0, TTC is valid and the collision time can be estimated. If the relative velocity v of the two objects r ≤ 0 (i.e., moving away from each other or having the same speed), TTC is usually considered meaningless. The smaller the TTC value, the higher the possibility and urgency of a collision. When TTC approaches 0, it means a collision is about to occur.

[0039] A further improvement of the present invention lies in: in step 4, setting different TTC thresholds to take corresponding early warning measures for the collision monitoring system, which specifically includes:

[0040] In the collision monitoring system designed by the present invention, different thresholds are set for the value of TTC, and the early warning is divided into three levels. The specific division formula is as follows:

[0041]

[0042] When TTC > 4.4 s, the warning system outputs 0, does not output the warning signal, and processes it as a local signal; when 2.8 s < TTC < 4.4 s, a first-level warning is output to remind the driver to pay attention to the road conditions; when 1.2 s < TTC < 2.8 s, a second-level warning is output, accompanied by an increasing reminder frequency; when TTC < 1.2 s, a third-level warning is output. The driving assistance system uses the vehicle's sensors to continuously sense the surrounding environment, warns of potential dangers, and directly controls the vehicle's deceleration or braking when necessary.

[0043] Beneficial effects: Compared with the prior art, the significant advantages of the present invention are as follows: Compared with the traditional vehicle trajectory prediction method that ignores individual driving style differences, the present invention considers the eye movement patterns of each driver and proposes a vehicle trajectory prediction method considering eye movement characteristics. By using the driver's eye movement characteristics as the model input, this method can reflect the driver's attention distribution and driving intention in advance, and is more forward-looking and real-time in trajectory prediction, achieving a higher-precision human-like vehicle trajectory prediction. Since the predicted vehicle trajectory is more human-like, this prediction method can greatly improve the accuracy of the collision monitoring system and reduce the driver's sense of rejection of the intelligent driving system. According to multiple experimental data, this method can be applied to general complex traffic scenarios. Compared with the traditional collision monitoring system, the present invention has an advance response of 0.5 seconds in the highway scenario; in the complex traffic scenario, the present invention performs more prominently, with an advance response of 1.5 seconds compared with the traditional collision monitoring system, which helps to efficiently and accurately monitor the vehicle collision hazards in the complex traffic scenario and remind the driver to take timely avoidance measures, promoting road traffic safety. Brief Description of the Drawings

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments are briefly introduced below. It should be noted that the following-described drawings only relate to some embodiments of the present invention, and those skilled in the art can derive other forms of drawings based on these drawings without creative efforts.

[0045] Figure 1 It is a flowchart of the intelligent driving collision monitoring method of the present invention;

[0046] Figure 2 It is a schematic diagram of the separating axis projection;

[0047] Figure 3 It is a schematic diagram of the vehicle collision angle;

[0048] Figure 4 It is a schematic diagram of the interaction between the target vehicle and the vehicle behind in the target lane. Detailed Embodiments

[0049] The following clearly and completely describes the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. It should be noted that the described embodiments are only a part of the embodiments of the present invention, rather than all possible embodiments. Based on these embodiments, all other implementation manners obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0050] The purpose of the present invention is to provide an intelligent driving collision monitoring method based on trajectory prediction, enabling the driver to timely obtain potential collision risk situations and take corresponding risk avoidance measures, thereby increasing the safety of road traffic.

[0051] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Figure 1 It is a flowchart of the intelligent driving collision monitoring method of the present invention; as Figure 1 shown, the present invention provides an intelligent driving collision monitoring method based on trajectory prediction, including the following steps:

[0053] Step 1: Use the following features of the host vehicle, including following characteristics, interaction behavior characteristics, and driver eye movement characteristics, as the input of the deep learning model to predict the vehicle's running trajectory;

[0054] Using the following features of the host vehicle, including following characteristics, interaction behavior characteristics, and driver eye movement characteristics, as the input of the deep learning model to predict the vehicle's running trajectory, the specific data feature indicators extracted include:

[0055] The lateral and longitudinal relative distances between the vehicle and the adjacent leading vehicle at time t;

[0056] The lateral and longitudinal speeds of the vehicle at time t;

[0057] The lateral and longitudinal speeds of the adjacent leading vehicle of the vehicle at time t;

[0058] The line-of-sight angles between the vehicle driver and the pedestrians on the left and right sides respectively at time t;

[0059] The lateral and longitudinal relative distances between the vehicle and the interacting pedestrian on the left at time t;

[0060] The lateral and longitudinal speeds of the interacting pedestrian on the left side of the vehicle at time t;

[0061] The lateral and longitudinal relative distances between the vehicle and the interacting pedestrian on the right at time t;

[0062] The lateral and longitudinal speeds of the interacting pedestrian on the right side of the vehicle at time t;

[0063] The driver's gaze position at time t;

[0064] The driver's gaze duration at time t;

[0065] Input the above extracted features into the LSTM-Attention model to obtain more accurate predicted trajectory points.

[0066] LSTM consists of structures such as input gates, forget gates, and output gates. The "gate" structure selectively controls the flow of information and can better capture time dependencies. The present invention uses the "gate" structure to mine decision-making and state features at historical moments, adaptively retain and forget historical information, and thus output a "memory vector", which can accurately reflect the driver's decision-making process in the gradually disappearing memory effect. The Attention mechanism is used to adaptively assign different weights to the spatio-temporal features of vehicle operation, enhance the model's attention ability to key spatio-temporal information, and more sensitively capture the dynamic changes of interaction features (such as relative direction, relative distance, line-of-sight angle, etc.), thereby improving the prediction accuracy and further enhancing the accuracy of subsequent collision warning analysis.

[0067] Step 2: After obtaining the coordinates of each vertex of the bounding box of the vehicle under test, select any one of the detection axes of the two vehicles as the projection axis, and detect whether the projections of the two vehicles overlap. Repeat the above steps until all four axes of the two vehicles are detected;

[0068] The oriented bounding box algorithm detects collisions based on the separating axis theorem. The separating axis theorem determines whether a collision occurs by judging whether the projections of two convex polygons overlap at any angle. The projection schematic diagram is as Figure 2 shown. In the actual scenario, the oriented bounding box algorithm replaces the actual complex vehicle shape with a simple rectangle that tightly encloses the vehicle under test. At this time, it is only necessary to project the bounding boxes of the two vehicles under test at four key angles to determine whether there is an overlap to achieve collision detection. The vehicle collision detection angles are as Figure 3 shown.

[0069] Assume that the length of the bounding box of the vehicle under test is 5.0m and the width is 2.0 meters. Based on existing research, improve the bounding box parameters with the traffic severity threshold as the warning lead time to achieve conflict detection and conflict severity detection. Let the length of the bounding box of the vehicle under test be L h , the width be K h , and T re be the warning lead time. The formula is as follows:

[0070] L h = 5 + v x *T re

[0071] K h = 2 + v x *T re

[0072] Taking the interaction between the target vehicle and the vehicle behind in the target lane as an example, the warning lead time is introduced to change the length of the bounding box of the vehicle behind in the target lane, so as to achieve the purpose of conflict detection, as Figure 4 shown. Figure 4 In it, the detection angle 1 is the driving direction of the vehicle behind in the target lane, the detection angle 2 is the direction perpendicular to the driving direction of the vehicle behind in the target lane, the detection lane 3 is the driving direction of the target vehicle, and the detection lane 4 is the direction perpendicular to the driving direction of the target vehicle. According to the predicted trajectory result in step 1, taking the severe conflict threshold as 0.6 s as an example, the coordinates of each vertex of the two bounding boxes are solved according to the following formula.

[0073]

[0074]

[0075] Among them, D1 - D4 are the vertices of vehicle 1; D5 - D8 are the vertices of vehicle 2; x D is the abscissa of the vertex; y D is the ordinate of the vertex; θ(t) is the direction angle of the target vehicle.

[0076] After obtaining the coordinates of each vertex of the vehicle, based on the separating axis theorem, it can be checked whether there is a conflict risk for the vehicle. The oriented bounding box algorithm detects collisions based on the separating axis theorem. The separating axis theorem determines whether a collision occurs by judging whether there is an overlap in the projections of two convex polygons at any angle. However, in the actual scenario, it is obviously not in line with the high timeliness required for collision detection to project all angles of the convex polygon. Therefore, the oriented bounding box algorithm replaces the actual complex vehicle shape with a simple rectangle that tightly encloses the vehicle to be measured. At this time, it is only necessary to project the bounding boxes of the two vehicles to be measured at four key angles to determine whether there is an overlap to achieve collision detection. If the projections at the four detection angles all overlap, the two vehicles collide. If any one of the projections does not overlap, there is no collision.

[0077] Step 3: Calculate the TTC through the predicted trajectory of the vehicle to judge the risk level;

[0078] The most likely accident type for vehicles in the traffic scenario is vehicle collision. TTC (Time to Collision) is an important traffic safety indicator used to evaluate the possibility and time of a collision between two traffic participants. It represents how long it will take for two vehicles or a moving object and a potential collision target to collide under the current motion state. The specific formula is as follows:

[0079]

[0080] Among them, d is the relative distance between the target vehicle and the vehicle in front; v ris the relative speed of the target vehicle and the vehicle ahead along the collision direction.

[0081] If two objects approach each other, v r > 0, TTC is valid and the collision time can be estimated. If the relative speed v of the two objects r ≤ 0 (i.e., moving away from each other or having the same speed), TTC is usually considered meaningless. The smaller the TTC value, the higher the probability and urgency of a collision. When TTC approaches 0, it means a collision is about to occur.

[0082] Step 4: Take different levels of warning measures for different degrees of risk conflicts.

[0083] In the collision monitoring system designed by the present invention, different thresholds are set for the value of TTC, and the warning is divided into three levels. The specific division formula is as follows:

[0084]

[0085] When TTC > 4.4 s, the warning system outputs 0, does not output a warning signal, and processes it as a local signal; when 2.8 s < TTC < 4.4 s, a first-level warning is output to remind the driver to pay attention to the road conditions; when 1.2 s < TTC < 2.8 s, a second-level warning is output, accompanied by an increasing reminder frequency; when TTC < 1.2 s, a third-level warning is output. The driving assistance system uses the vehicle's sensors to continuously sense the surrounding environment, warns of potential dangers, and directly controls the vehicle's deceleration or braking when necessary.

Claims

1. An intelligent driving collision monitoring method based on trajectory prediction, characterized in that, It includes the following steps: Step 1: Use the following features of the host vehicle, i.e., vehicle following characteristics, interaction behavior characteristics, and driver eye movement characteristics, as the input of the LSTM-Attention deep learning model to predict the vehicle running trajectory; Step 2: After obtaining the coordinates of each vertex of the bounding box of the vehicle under test, select any one of the detection axes of the two vehicles as the projection axis, and detect whether the projections of the two vehicles overlap. Repeat the above steps until all four axes of the two vehicles are detected; Step 3: Calculate the TTC (Time to Collision) through the predicted vehicle trajectory to judge the risk level; Step 4: Take different levels of warning measures for different levels of risk conflicts.

2. The intelligent driving collision monitoring method based on trajectory prediction according to claim 1, wherein, In the above Step 1, the vehicle following characteristics of the host vehicle include dynamic parameters such as vehicle speed, acceleration, and distance from the vehicle ahead; the interaction behavior characteristics include the lane-changing behavior, acceleration and deceleration behavior of surrounding vehicles, and the relative position and relative speed with pedestrians crossing the street and non-motor vehicles; the driver eye movement characteristics include the line-of-sight fixation point, saccade amplitude, line-of-sight angle, and blink frequency, etc. The LSTM-Attention model is used to jointly model the above characteristics through multi-modal data fusion technology to predict the vehicle running trajectory. Among them, the attention weights in different visual fields of the driver are adjusted in real time and dynamically according to the collected driver eye movement characteristics, and a vehicle running trajectory prediction model with higher accuracy and human-like characteristics suitable for general complex traffic scenarios is constructed.

3. The intelligent driving collision monitoring method based on trajectory prediction according to claim 1, characterized in that, In the above Step 2, the Oriented Bounding Box (OBB) method is used to calculate the bounding box of the vehicle, and the shape of the bounding box is dynamically adjusted according to the length, width, and angle information of the vehicle, and the projection detection method is combined to accurately judge the spatial conflict situation between the two vehicles.

4. The intelligent driving collision monitoring method based on trajectory prediction according to claim 1, characterized in that, In the above Step 3, by calculating the TTC value, based on the intersection situation of the predicted vehicle trajectory at different time points, the time threshold when a collision may occur is determined, and the risk level is quantitatively analyzed in combination with the vehicle dynamic characteristics.

5. A method for intelligent driving collision monitoring based on trajectory prediction according to claim 1, characterized in that, In the above Step 4, the warning measures include multi-modal feedback forms such as vision, audition, and touch; for low-risk conflicts, visual cues are mainly used; for medium-risk conflicts, auditory alarm cues are added; For high-risk conflicts, active intervention measures such as steering wheel vibration or emergency braking are further triggered.

Images