Multi-level automatic driving forward collision safety early warning system and method

Abstract

The invention discloses a multi-level automatic driving forward collision safety early warning system and method, and relates to the technical field of intelligent traffic and traffic safety. The automatic driving forward collision safety pre-warning system is composed of a data information acquisition unit, a collision pre-warning calculation unit and a man-machine interaction reminding unit, and comprises a sensing information acquisition unit, an emergency state judgment unit, a pre-warning result judgment unit, a takeover information prompt unit and a driving operation control unit. By establishing a forward collision safety early warning algorithm and designing a man-machine reminding interaction mode combination mode based on vision, hearing and touch, in the system, when an automatic driving vehicle has an emergency or exceeds a system operation area, taking-over information is clearly sent to a driver, and the defects of automatic driving aided driving at the present stage are effectively overcome.

Description

Technical Field

[0001] The present invention relates to the field of intelligent driving information, and in particular to a multi-level automatic driving forward collision safety warning system and method. Background Art

[0002] With the continuous application of intelligent network technology in the automotive industry and the continuous breakthroughs in artificial intelligence, control algorithms and vehicle-mounted sensors, autonomous driving technology has developed rapidly, and various driving assistance systems have been put into use. At present, the development of autonomous driving technology is not mature, and traffic accidents caused by autonomous driving occur from time to time. The reason is that some conditional autonomous driving vehicles allow the driver to temporarily transfer the driving task to the vehicle. In an emergency, the driver needs to take over the autonomous driving vehicle in time, but the driver will fall into a passive fatigue state if he is away from the driving task for a long time, thereby reducing the efficiency of taking over. In this case, traffic accidents are very likely to occur.

[0003] Therefore, there is an urgent need for a safety assurance measure applied to multi-level autonomous driving to reduce the occurrence of autonomous driving traffic accidents. The autonomous driving forward collision safety warning system and takeover method proposed in the present invention, through a collision warning algorithm, judges the degree of danger of the driving environment, and can meet the advance takeover warning under different vehicle-road conditions from multiple human-vehicle interaction system modes such as vision, hearing, and touch, so that the driver can accurately identify the takeover request, thereby avoiding the occurrence of emergency events that cause the driver to panic or collision accidents, and improving traffic driving safety. It effectively conforms to the development needs of future intelligent transportation systems (autonomous driving) and has a good effect on maintaining traffic order and reducing traffic accidents. Summary of the invention

[0004] The purpose of the present invention is to provide a multi-level automatic driving forward collision safety warning system and method, which integrates and processes the collected data from each system, establishes a forward collision safety warning model, lists the corresponding reminder takeover emergency level, and converts the obtained results into output warning results, which are finally fed back to the driver, ensuring that the driver can accurately, quickly and promptly receive the warning signal, and be guided to make corresponding driving operations, fully ensuring the driving safety of automatic driving.

[0005] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is:

[0006] A multi-level automatic driving forward collision safety warning system and method, comprising a data information acquisition unit, a collision warning calculation unit and a human-computer interaction reminder unit;

[0007] After the data information acquisition unit collects data, the collision warning calculation unit performs data information fusion processing on the corresponding data. The human-computer interaction reminder unit receives the instruction from the collision warning calculation unit and issues the corresponding level of warning information according to the set warning degree and level definition.

[0008] When receiving the alarm information from the vehicle, the driver is reminded to take measures to avoid collision. If the collision risk is eliminated, the warning prompt is cancelled. If the driver is in poor condition and cannot take over the vehicle and take braking measures, or the driver does not take over the vehicle, the automatic driving system will automatically enter the minimum risk control mode, gradually reduce the speed, and ensure that the vehicle brakes to a stop before a collision occurs;

[0009] The data information fusion processing is to fuse the data collected by the data information collection unit, and the content of the fusion processing includes output layer, modal feature extraction, and feature fusion;

[0010] The input layer inputs the video data and radar data into the model. The video data uses a deep learning network to extract image features, and the radar data uses a Chinese BEV conversion network to extract point cloud features.

[0011] The modal feature extraction includes video modal feature extraction and radar modal feature extraction; video modal feature extraction uses FasterR-CNN to extract the spatial features of the target in the video and outputs a two-dimensional feature map; radar modal feature extraction uses PointNet++ to extract local and global geometric features and output a two-dimensional or three-dimensional feature tensor. The output content is:

[0012] F image =f CNN (I);

[0013] F radar =f PointNet (P);

[0014] Among them, F image represents the features extracted from the video data, F rader represents the features extracted from radar data, f CNN It means that the convolutional neural network extracts high-dimensional features of the image through convolutional layers, pooling layers and nonlinear activation functions, I represents the image data in the video data, and f PointNet represents a network for processing point cloud data, and P represents radar point cloud data;

[0015] The feature fusion described above requires aligning the features of different modalities into a unified reference coordinate system through a modality alignment mechanism;

[0016] Radar data is projected from the radar coordinate system to the image plane through the camera calibration matrix to establish the spatial relationship between the radar coordinate system and the camera coordinate system;

[0017] The image data in the video data uses stereo vision or structured light technology to obtain the depth information in the video. Each pixel in the image data can be mapped to a three-dimensional space to obtain the depth value of the pixel. These depth information converts the video data from a two-dimensional image to a point cloud in a three-dimensional space, so that the video data and radar data are located in the same three-dimensional coordinate space for subsequent feature fusion. Finally, the feature tensor is convolved point by point to achieve deep fusion. The final fusion feature is as follows:

[0018] F fused =f CNN ([F image ,F radar ]);

[0019] Among them, F fused represents the final fusion feature after being processed by the CNN network, [F image ,F rader ] means concatenating the video and radar features together to form a new feature vector.

[0020] Preferably, the data information acquisition unit uses a speed sensor to collect relevant speed data such as the speed and acceleration of the vehicle, and uses a camera and millimeter radar waves to collect the speed, position and azimuth angle data of the forward vehicle;

[0021] The collision warning calculation unit performs data information fusion processing on the data information collection unit, and combines the driving state information of the vehicle to execute a certain collision warning algorithm to calculate and evaluate whether there is a potential collision safety risk. If it is determined that there is a risk, a warning instruction is issued to the human-computer interaction reminder unit;

[0022] After point-by-point convolution operation, deep fusion is achieved. Deep fusion includes early fusion and late fusion. Early fusion is to splice the feature maps of video images and radar data together at the initial stage of data, so that video features and radar features are used as parallel inputs and passed into a shared convolution layer to merge the feature maps of the two modalities into a large feature map. Late fusion is that the features of video and radar data are processed by their own convolution layers respectively, so that video and radar data can learn features independently in different channels, and then fuse them through specific operations.

[0023] The deeply fused feature map is further processed through several convolutional layers to gradually extract more advanced and abstract features. During the feature extraction process, the convolutional neural network automatically learns the features and reduces redundant information.

[0024] The driver status is that the automatic driving system monitors the driver's driving behavior at all times and determines whether the driver has the ability to take over the vehicle by monitoring the driver's blinking frequency in the last 10 seconds.

[0025] Preferably, a forward collision warning decision algorithm based on collision time and minimum safe braking distance is constructed in the collision warning calculation unit, and the forward collision warning decision algorithm includes warning scenario analysis, warning strategy design and driving status discussion;

[0026] The warning scenario analysis includes:

[0027] A1. Establishing Gaussian plane coordinate system: transform the latitude and longitude information of the vehicle and the preceding vehicle into position coordinates in the Gaussian plane coordinate system to obtain the center coordinates of the vehicle, which is convenient for subsequent calculation of vehicle safety collision warning;

[0028] A2. Vehicle collision type: In actual driving, there is no need to issue collision warnings to all vehicles. By classifying vehicles that may collide, corresponding warning decisions and reminder methods can be effectively applied to different collision types. Through the Gaussian plane coordinate system, the vehicle coordinates and heading angle information can be accurately obtained, and the possible collision types can be divided into forward collision and rear-end collision.

[0029] The early warning strategy design includes:

[0030] B1. Collision strategy based on TTC: Based on the analysis of the relative speed between the vehicle and the vehicle in front, the relative collision event is calculated, and the warning time threshold is set to perform warning classification processing on the collision risk; TTC is a measure of the time required for a collision between a vehicle and an obstacle;

[0031] B2. Warning strategy based on minimum safe braking distance: Calculate the minimum warning distance based on the speed of the vehicle and the speed of the preceding vehicle, and compare it with the actual target distance to formulate a safe braking distance collision warning strategy;

[0032] B3. Coupling warning strategy based on TTC and minimum safe braking distance: When the relative speed difference between the vehicle and the preceding vehicle is small and the actual detection distance is less than the safety set distance, the collision warning method based on the minimum safe braking distance is likely to trigger a high-level warning, while the actual risk does not require a warning, resulting in a false alarm. The warning strategy based on the collision time coupled with the minimum safe braking distance can effectively improve the accuracy of the collision warning.

[0033] The driving status discussion includes: establishing a forward collision warning model for different driving states of the preceding vehicle to meet different traffic operating environments, including a stationary state, a uniform speed state, a uniform deceleration state, a uniform acceleration state and a maximum braking state.

[0034] Preferably, the analysis process of the warning scenario specifically includes the following steps:

[0035] SA1. Constructing Gaussian plane rectangular coordinate system: When calculating the relative position distance between vehicles and constructing the vehicle model, the Gauss-Rüger projection technology is required to convert the initially received latitude and longitude coordinates (P, L) into more intuitive Gaussian plane coordinates (x, y), laying the foundation for the establishment of the subsequent safety collision model. The vehicle position coordinates (x i ,y i ) can be achieved by following the steps below:

[0036]

[0037] Where: x i is the abscissa of the Gaussian coordinate system; y i is the ordinate of the Gaussian coordinate system; X0 is the length of the meridian arc from the equator to the parallel circle of latitude D; l is the difference in degrees between the longitude L of the projection point and the meridian longitude L0 of the longitude zone axis;

[0038] The calculation method of the data in the formula is: l = L-L0; L0 = 6n-3, n = round [(L+3) / 6]; N is the radius of curvature of the yoke; t = tan D, ζ = e'cosD;

[0039] Where: X0 = (1-e 2 )(R0D+R1sin2D+R2sin4D+R3sin6D+R4sin8D);

[0040] Where: R0, R1, R2, R3, R4 are parameters, and their specific meanings are as follows:

[0041]

[0042] Where: m = 0.0033528; e = 0.0818192;

[0043] SA2. Obtaining vehicle collision types: Classify vehicles that may collide, and accurately obtain the coordinates and heading angle information of the vehicles through the constructed Gaussian plane coordinate system. According to the driving directions of the self-vehicle and the front vehicle, the possible collision types are divided into forward collision and rear-end collision. When the self-vehicle and the front vehicle are driving towards each other, there may be a forward collision; when the self-vehicle and the front vehicle are driving in the same direction, there may be a rear-end collision.

[0044] The collision judgment condition is:

[0045] In the formula, It is the angle between the line connecting the center coordinates of the vehicle and the front vehicle and the driving direction of the vehicle. The angle is the relative heading angle; the angle threshold Δ=10°.

[0046] Preferably, the design of the early warning strategy includes the following steps:

[0047] SB1. Construct a collision warning strategy based on TTC: Based on the relationship between the relative driving status of the vehicles and the distance required for following braking, a collision time warning model is established; when the time to an impending collision reaches the preset warning time threshold, the system will immediately issue a corresponding prompt message; the calculation of TTC follows the following formula:

[0048]

[0049] Where: Δd is the relative distance between the vehicle and the front vehicle; Δv is the relative speed of the two vehicles; v n is the vehicle speed; v m is the speed of the vehicle in front; the risk of a collision between the two vehicles is inversely proportional to the TTC value. The smaller the TTC value, the greater the risk of a collision. This collision warning strategy can show high reliability when the vehicle is traveling at a low speed;

[0050] SB2, warning strategy based on minimum safe braking distance: Calculate the minimum warning distance based on the speed of the vehicle and the speed of the preceding vehicle, and compare it with the actual target distance, based on which a safe braking distance collision warning strategy is formulated;

[0051] In the vehicle's braking process, the safe collision avoidance time includes the driver's reaction time t1 from identifying danger to starting braking; the coordination time t2 from the start of braking with the brake pedal to the braking effect, t2 is set to 0.2 seconds; the growth time t3 from the braking effect to the hydraulic brake generating braking action, and the duration of no braking force t4;

[0052] The specific calculation formula is as follows:

[0053] Assuming the initial speed of the vehicle is v0, the distance d1 traveled during the time period t1 is:

[0054] d1=v0t1;

[0055] The travel distance d2 during the time period t2 is:

[0056]

[0057] In the formula, a max is the maximum deceleration, in m / s 2 ;

[0058] The travel distance d3 during the time period t3 is:

[0059]

[0060] The total braking distance D of the vehicle is obtained as:

[0061]

[0062] During the braking process of the vehicle, the calculation formula for the duration t4 without braking force is:

[0063]

[0064] In the formula, v is the vehicle speed; g is the acceleration of gravity, λ is the road adhesion coefficient, and the maximum deceleration a is max =λg;

[0065] Specifically, the total braking distance D of the vehicle is:

[0066]

[0067] SB3. Coupling warning strategy based on TTC and minimum safe braking distance: The collision warning calculation formula based on the coupling of collision time TTC and minimum safe braking distance is as follows:

[0068]

[0069] Preferably, the driving status discussion specifically includes the following steps:

[0070] SC1. Build a corresponding forward collision warning model for various driving states that the vehicle ahead may present, including stationary, constant speed, constant deceleration, constant acceleration and maximum braking state, and calculate the collision time TTC value under each state to ensure that the system can adapt to different traffic operating environments;

[0071] SC2. Calculate the TTC value in each state:

[0072] When the vehicle is stationary, v m =0, then:

[0073] When the current vehicle is at a constant speed, v m >0, a m =0, then:

[0074] When the current vehicle is in a uniform deceleration state, m g<a m <0, then:

[0075] When the current vehicle is in a uniform acceleration state, a m >0, then:

[0076] When the vehicle is in the maximum braking state, a m =λ m g, then:

[0077] Preferably, the automatic driving includes a safety warning system for human-vehicle takeover, and the safety warning system includes a perception information acquisition unit, an emergency state judgment unit, a warning result judgment unit, a takeover information prompt unit, and a driving operation control unit;

[0078] The perception information acquisition unit integrates the roadside unit and the vehicle-mounted sensors to collect the vehicle's speed, position, acceleration and surrounding environment status information in real time;

[0079] The emergency state judgment unit estimates the vehicle driving time based on the perception information data, calculates the remaining time for automatic driving judgment, and confirms the reminder type of the human-vehicle takeover warning;

[0080] The warning result judgment unit generates different types of system warning judgment results based on the relationship between the remaining driving time and the set warning time threshold;

[0081] The takeover information prompt unit sends corresponding takeover prompts to the driver according to the different levels of the warning results. The intensity of the prompt method will also vary according to the severity of the warning.

[0082] After the takeover warning is triggered, the driving operation control unit is responsible for achieving a smooth transition between the automatic driving mode and the manual control mode and making corresponding driving status adjustments.

[0083] Preferably, it is characterized in that the emergency state judgment unit specifically includes:

[0084] Warning time thresholds, the warning time thresholds include a first warning time threshold, a second warning time threshold, a third warning time threshold and a fourth warning time threshold, the first warning time threshold is greater than the second warning time threshold, the second warning time threshold is greater than the third warning time threshold, and the third warning time threshold is greater than the fourth warning time threshold;

[0085] System warning judgment results, the system warning judgment results include no system warning results, the first type of system warning judgment results, the second type of system warning judgment results, the third type of system warning judgment results and the fourth type of system warning judgment results.

[0086] Preferably, when the vehicle determines the remaining time, the relationship between the remaining time and the warning time threshold is determined as follows:

[0087] When the vehicle determines that the remaining time is greater than the first warning time threshold, the no system warning result occurs;

[0088] When the vehicle judged remaining time is less than or equal to the first warning time threshold but greater than the second warning time threshold, the first type of system warning judgment result appears;

[0089] When the vehicle judges that the remaining time is less than or equal to the second warning time threshold but greater than the third warning time threshold, the second type of system warning judgment result appears;

[0090] When the vehicle judged remaining time is less than or equal to the third warning time threshold but greater than the fourth warning time threshold, the third type of system warning judgment result appears;

[0091] When the vehicle judged remaining time is less than or equal to the fourth warning time threshold, the fourth type of system warning judgment result appears;

[0092] Preferably, the takeover warning prompting method includes a takeover reminder based on visual interaction, a takeover reminder based on auditory interaction, and a takeover reminder based on tactile interaction;

[0093] The types of warning information provided by the early warning include:

[0094] When responding to the no-takeover prompt message, the system will warn that there is no change in the color or flashing frequency of the dashboard LED light, and there is no change in the alarm sound or vibration;

[0095] When responding to the first type of takeover prompt information, the system warning prompt includes a blue LED light strip on the instrument panel, a flashing frequency of 0-1 times / second, and no alarm sound or vibration frequency change;

[0096] When responding to the second type of takeover prompt information, the system warning prompt includes a green LED light strip on the instrument panel, a flashing frequency of 0-1 times / second, a voice alarm sound, and a vibration frequency f<10Hz;

[0097] When responding to the third type of takeover prompt information, the system warning prompt includes an orange LED light strip on the instrument panel, a flashing frequency of 1-2 times / second, a voice alarm sound, and a vibration frequency of 10Hz≤f≤200Hz;

[0098] When responding to the fourth type of takeover prompt information, the system warning prompt includes a red LED light strip on the instrument panel, a flashing frequency of 2-4 times / second, an alarm type alarm sound, and a vibration frequency f>200Hz;

[0099] When the fourth warning time threshold is about to expire and the driver has not taken over the vehicle: relative heading angle Turn on the left turn signal, and the vehicle control system controls the vehicle to change lanes to the left and slow down, with a left turning angle of Δ;

[0100] When the relative heading angle , turn on the right turn signal, and the vehicle control system controls the vehicle to change lanes to the right and slow down, with a right turning angle of Δ.

[0101] The beneficial effects of the present invention are:

[0102] (1) The multi-level automatic driving forward collision safety warning system and multi-mode takeover method proposed in the present invention can construct a corresponding forward collision warning model according to the different driving states of the vehicle, which is more in line with the actual collision scenario, thereby effectively avoiding collision accidents and improving driving safety.

[0103] (2) The present invention proposes a multi-level automatic driving forward collision safety warning system and a multi-mode takeover method. By prompting the remaining driving time and the warning time threshold, and performing graded processing on the system warning judgment results and takeover prompt information, it is more intelligent and reasonable, and effectively ensures the timeliness and safety of the driver's takeover.

[0104] (3) The present invention proposes a multi-level automatic driving forward collision safety warning system and a multi-mode takeover method, which supports multi-modal human-vehicle interaction based on vision, hearing, and touch, and can provide the driver with rich interactive information in a short time, meeting the early takeover warning needs under different road and traffic conditions, and has a strong processing speed advantage. BRIEF DESCRIPTION OF THE DRAWINGS

[0105] Figure 1 This is a diagram of the forward collision warning system.

[0106] Figure 2 It is a schematic diagram of the vehicle braking process.

[0107] Figure 3 It is a coupled warning flow chart based on collision time and minimum safe braking distance.

[0108] Figure 4 It is a schematic diagram of the system prompt unit module composition.

[0109] Figure 5 It is a schematic diagram of a safety early warning system and method.

[0110] Figure 6 It is a schematic diagram of prompt takeover based on the visual and auditory interaction system model.

[0111] Figure 7 It is a schematic diagram of prompt takeover based on the tactile interaction system mode.

[0112] Figure 8It is a flow chart of a safety warning system and method for automatic driving human-vehicle takeover. DETAILED DESCRIPTION

[0113] The present invention is described in detail below in conjunction with the accompanying drawings:

[0114] Example 1

[0115] Combination Figures 1 to 8 A multi-level automatic driving forward collision safety warning system and method includes a data information acquisition unit, a collision warning calculation unit and a human-computer interaction reminder unit. After the data information acquisition unit performs data acquisition, the collision warning calculation unit performs data information fusion processing on the corresponding data. The human-computer interaction reminder unit receives the instruction from the collision warning calculation unit and issues the corresponding level of warning information according to the set warning degree and level definition.

[0116] When receiving the alarm information from the vehicle, the driver is reminded to take measures to avoid collision. If the collision risk is eliminated, the warning prompt is cancelled. If the driver is in poor condition and cannot take over the vehicle and take braking measures, or the driver does not take over the vehicle, the automatic driving system will automatically enter the minimum risk control mode, gradually reduce the speed, and ensure that the vehicle brakes to a stop before a collision occurs;

[0117] Data information fusion processing is to fuse the data collected by the data information acquisition unit. The fusion processing content includes output layer, modal feature extraction, and feature fusion; the input layer is to input video data and radar data into the model. The video data uses a deep learning network to extract image features, and the radar data uses a Chinese BEV conversion network to extract point cloud features.

[0118] Modal feature extraction includes video modal feature extraction and radar modal feature extraction; video modal feature extraction uses FasterR-CNN to extract the spatial features of the target in the video and outputs a two-dimensional feature map; radar modal feature extraction uses PointNet++ to extract local and global geometric features and output a two-dimensional or three-dimensional feature tensor. The output content is: F image =f CNN (I); F radar =f PointNet (P);

[0119] Among them, F image represents the features extracted from the video data, F rader represents the features extracted from radar data, f CNN It means that the convolutional neural network extracts high-dimensional features of the image through convolutional layers, pooling layers and nonlinear activation functions, I represents the image data in the video data, and f PointNet represents a network for processing point cloud data, and P represents radar point cloud data;

[0120] Feature fusion requires aligning features of different modalities into a unified reference coordinate system through a modal alignment mechanism; radar data is achieved by projecting points in the radar coordinate system onto the image plane through a camera calibration matrix to establish a spatial relationship between the radar coordinate system and the camera coordinate system; image data in video data uses stereo vision or structured light technology to obtain depth information in the video. Each pixel in the image data can be mapped to three-dimensional space to obtain the depth value of the pixel. These depth information converts the video data from a two-dimensional image to a point cloud in three-dimensional space.

[0121] Therefore, the video data and radar data are located in the same three-dimensional coordinate space for subsequent feature fusion. Finally, the feature tensor is convolved point by point to achieve deep fusion. The final fusion feature is: fused =f CNN ([F image ,F radar ]); where F fused represents the final fusion feature after being processed by the CNN network, [F image ,F rader ] means concatenating the video and radar features together to form a new feature vector.

[0122] The data information acquisition unit uses a speed sensor to collect relevant speed data such as the vehicle's speed and acceleration, and uses a camera and millimeter radar waves to collect the speed, position and azimuth data of the forward vehicle; the collision warning calculation unit performs data information fusion processing on the data information collected by the data information acquisition unit, and combines the vehicle's driving status information to execute a certain collision warning algorithm to calculate and evaluate whether there is a potential collision safety risk. If it is determined that there is a risk, a warning instruction is issued to the human-computer interaction reminder unit.

[0123] After point-by-point convolution operation, deep fusion is achieved. Deep fusion includes early fusion and late fusion. Early fusion is to splice the feature maps of video images and radar data together at the initial stage of data, so that video features and radar features are used as parallel inputs and passed into a shared convolution layer to merge the feature maps of the two modalities into a large feature map. Late fusion is that the features of video and radar data are processed by their own convolution layers respectively, so that video and radar data can learn features independently in different channels, and then fuse them through specific operations.

[0124] The deeply fused feature map is further processed through several convolutional layers to gradually extract more advanced and abstract features. In the process of feature extraction, the convolutional neural network will automatically learn features and reduce redundant information. The driver status is the automatic driving system's constant monitoring of the driver's driving behavior. By monitoring the driver's blinking frequency in the last 10 seconds, it can be used to determine whether the driver has the ability to take over the vehicle.

[0125] In the collision warning calculation unit, a forward collision warning decision algorithm based on collision time and minimum safe braking distance is constructed. The forward collision warning decision algorithm includes warning scenario analysis, warning strategy design and driving status discussion. The warning scenario analysis content includes: A1. Establishing a Gaussian plane coordinate system: converting the latitude and longitude information of the vehicle and the front vehicle into position coordinates in the Gaussian plane coordinate system to obtain the center coordinates of the vehicle, which is convenient for subsequent vehicle safety collision warning calculations; A2. Vehicle collision type: In actual vehicle driving, there is no need to issue collision warning reminders to all vehicles. By classifying vehicles that may collide, corresponding warning decisions and reminder methods can be effectively applied to different collision types. Through the Gaussian plane coordinate system, the vehicle coordinates and heading angles and other information can be accurately obtained, and the possible collision types can be divided into forward collisions and rear-end collisions.

[0126] The design of the warning strategy includes: B1. Collision strategy based on TTC: by analyzing the relative speed between the ego vehicle and the vehicle in front, the relative collision event is calculated, and the warning time threshold is set to perform warning classification processing on the collision risk; TTC is a measure of the time required for a collision between a vehicle and an obstacle; B2. Warning strategy based on minimum safe braking distance: the minimum warning distance is calculated based on the speed of the ego vehicle and the speed of the vehicle in front, and compared with the actual target distance, a safe braking distance collision warning strategy is formulated; B3. Warning strategy based on the coupling of TTC and minimum safe braking distance: when the relative speed difference between the ego vehicle and the vehicle in front is small, and when the actual detection distance is less than the safe set distance, the collision warning method based on the minimum safe braking distance is prone to trigger a high-level warning, while the actual risk does not require a warning, resulting in a false alarm. The warning strategy based on the collision time coupling minimum safe braking distance can effectively improve the accuracy of collision warnings.

[0127] The discussion on driving status includes: establishing a forward collision warning model for different driving states of the vehicle ahead to meet different traffic operating environments, including stationary state, uniform speed state, uniform deceleration state, uniform acceleration state and maximum braking state.

[0128] The analysis process of the warning scene specifically includes the following steps: SA1. Constructing the Gaussian plane rectangular coordinate system: When calculating the relative position distance between vehicles and constructing the vehicle model, the Gauss-Rüger projection technology is required to convert the initially received latitude and longitude coordinates (P, L) into more intuitive Gaussian plane coordinates (x, y), laying the foundation for the establishment of the subsequent safety collision model. The vehicle's position coordinates (x i ,y i ) can be achieved by following the steps below:

[0129]

[0130] Where: x i is the abscissa of the Gaussian coordinate system; y i is the ordinate of the Gaussian coordinate system; X0 is the length of the meridian arc from the equator to the parallel circle of latitude D; l is the difference in degrees between the longitude L of the projection point and the meridian longitude L0 of the longitude zone axis; the data is calculated as follows: l=L-L0; L0=6n-3, n=round[(L+3) / 6]; N is the radius of curvature of the equator; t=tan D, ζ=e′cos D.

[0131] in:

[0132] X0=(1-e 2 )(R0D+R1sin2D+R2sin4D+R3sin6D+R4sin8D);

[0133] Where: R0, R1, R2, R3, R4 are parameters, and their specific meanings are as follows:

[0134]

[0135] Where: m = 0.0033528; e = 0.0818192;

[0136] SA2. Obtaining vehicle collision types: Classify vehicles that may collide, and accurately obtain the coordinates and heading angle information of the vehicles through the constructed Gaussian plane coordinate system. According to the driving directions of the self-vehicle and the front vehicle, the possible collision types are divided into forward collision and rear-end collision. When the self-vehicle and the front vehicle are driving towards each other, there may be a forward collision; when the self-vehicle and the front vehicle are driving in the same direction, there may be a rear-end collision.

[0137] The collision judgment condition is:

[0138] In the formula, It is the angle between the line connecting the center coordinates of the vehicle and the front vehicle and the driving direction of the vehicle. The angle is the relative heading angle; the angle threshold Δ=10°.

[0139] The design of the warning strategy has the following steps:SB1. Construct a collision warning strategy based on TTC: According to the relationship between the relative driving state between vehicles and the required distance for following braking, a collision time warning model is established; when the time of impending collision reaches the preset warning time threshold, the system will immediately issue a corresponding prompt message; Among them, the calculation of TTC follows the following formula:

[0140] Where: Δd is the relative distance between the vehicle and the front vehicle; Δv is the relative speed of the two vehicles; v n is the vehicle speed; v m is the speed of the vehicle in front; the risk of a collision between the two vehicles is inversely proportional to the TTC value. The smaller the TTC value, the greater the risk of a collision. This collision warning strategy can show high reliability when the vehicle is traveling at a low speed; SB2, warning strategy based on minimum safe braking distance: calculate the minimum warning distance based on the speed of the vehicle and the speed of the vehicle in front, and compare it with the actual target distance, and formulate a safe braking distance collision warning strategy based on this;

[0141] In the vehicle's braking process, the safe collision avoidance time includes the driver's reaction time t1 from identifying danger to starting braking; the coordination time t2 from the start of braking with the brake pedal to the braking effect, t2 is set to 0.2 seconds; the growth time t3 from the braking effect to the hydraulic pressure generating braking action, and the duration t4 of no braking force.

[0142] The specific calculation formula is as follows:

[0143] Assuming the initial speed of the vehicle is v0, the distance d1 traveled in the time period t1 is: d1 = v0t1;

[0144] The travel distance d2 during the time period t2 is: In the formula, a max is the maximum deceleration, in m / s 2 ;

[0145] The travel distance d3 during the time period t3 is:

[0146] The total braking distance D of the vehicle is obtained as:

[0147] During the braking process of the vehicle, the calculation formula for the duration t4 without braking force is: In the formula, v is the vehicle speed; g is the acceleration of gravity, λ is the road adhesion coefficient, and the maximum deceleration a is max =λg; the total braking distance D of the vehicle is:

[0148] SB3. Coupling warning strategy based on TTC and minimum safe braking distance: The collision warning calculation formula based on the coupling of collision time TTC and minimum safe braking distance is as follows:

[0149] The discussion of driving status specifically includes the following steps: SC1. Construct a corresponding forward collision warning model for various driving states that the front vehicle may present, including stationary, constant speed, constant deceleration, constant acceleration and maximum braking state, and calculate the collision time TTC value under each state to ensure that the system can adapt to different traffic operating environments; SC2. Calculate the TTC value in each state:

[0150] When the vehicle is stationary, v m =0, then:

[0151] When the current vehicle is at a constant speed, v m >0, a m =0, then:

[0152] When the current vehicle is in a uniform deceleration state, m g<a m <0, then:

[0153] When the current vehicle is in a uniform acceleration state, a m >0, then:

[0154] When the vehicle is in the maximum braking state, a m =λ m g, then:

[0155] The autonomous driving includes a safety warning system for human-vehicle takeover, which includes a perception information acquisition unit, an emergency state judgment unit, a warning result judgment unit, a takeover information prompt unit and a driving operation control unit; the perception information acquisition unit collects the vehicle's speed, position, acceleration and surrounding environment status information in real time by integrating the roadside unit and the vehicle-mounted sensor; the emergency state judgment unit estimates the vehicle's driving time based on the perception information data, calculates the remaining time for autonomous driving judgment, and confirms the reminder type of the human-vehicle takeover warning. The warning result judgment unit generates different types of system warning judgment results based on the relationship between the remaining driving time and the set warning time threshold; the takeover information prompt unit issues corresponding takeover prompts to the driver according to the different levels of the warning results, and the intensity of the prompt method will vary according to the severity of the warning; after the takeover warning is triggered, the driving operation control unit is responsible for achieving a smooth transition between the autonomous driving mode and the manual control mode and making corresponding driving state adjustments.

[0156] Example 2

[0157] According to the "Automobile Driving Automation Classification Standard", based on the degree to which the driving automation system can perform dynamic driving tasks, according to the role allocation in the execution of dynamic driving tasks and whether there are design operating conditions restrictions, the level of autonomous driving is divided into Level 0-Level 5, and the higher the level of autonomous driving, the higher the level of vehicle automation. With the continuous advancement of intelligent technologies such as the Internet and big data, the current level of vehicle automation is Level 2-Level 3 due to the limitations of infrastructure and other technologies. When the vehicle exceeds the designed scope of use during driving, the autonomous driving function is affected, which requires the driver to provide corresponding assisted driving functions. In order to remind and warn the driver intuitively and quickly, a combination of lights, sounds and vibrations is required to achieve the desired effect.

[0158] This embodiment provides a multi-level automatic driving forward collision safety warning system and a multi-mode takeover method, which is characterized in that the automatic driving forward collision safety warning system and the takeover method are composed of main units such as data information collection, collision warning calculation and human-computer interaction reminder, such as Figure 1 As shown. The data information acquisition unit uses the speed sensor to collect data such as the speed and acceleration of the vehicle, and uses the camera and millimeter radar wave to collect data such as the speed, position and azimuth of the front vehicle; the collision warning calculation unit integrates and processes the data information collected from the front vehicle, and combines the driving status information of the vehicle to execute a certain collision warning algorithm to calculate and evaluate whether there is a potential collision safety risk. If it is judged that there is a risk, a warning instruction is issued to the human-computer interaction reminder unit.

[0159] The human-computer interaction reminder unit receives instructions from the collision warning calculation unit, and issues corresponding level warning information according to the set warning degree and level definition. When receiving the alarm information issued by the vehicle itself, it reminds the driver to take measures to avoid collision. If the collision risk is eliminated, the warning prompt is cancelled.

[0160] Construct a forward collision warning decision algorithm based on time to collision (TTC) and minimum safe braking distance, including warning scenario analysis, warning strategy design, and driving status discussion;

[0161] The warning scenario analysis includes: (1) establishing a Gaussian plane coordinate system: converting the longitude and latitude information of the vehicle and the preceding vehicle into position coordinates in the Gaussian plane coordinate system to obtain the center coordinates of the vehicle, which is convenient for subsequent calculation of vehicle safety collision warning; (2) vehicle collision type: in actual vehicle driving, there is no need to issue collision warning reminders to all vehicles. By classifying vehicles that may collide, corresponding warning decisions and reminder methods can be effectively applied to different collision types. Through the established Gaussian plane coordinate system, information such as vehicle coordinates and heading angles can be accurately obtained, and possible collision types can be divided into head-on collision and rear-end collision.

[0162] The warning strategy design includes: (1) Collision strategy based on TTC: by analyzing the relative speed between the ego vehicle and the vehicle in front, the relative collision event is calculated, and the warning time threshold is set to perform warning classification processing on the collision risk; (2) Warning strategy based on minimum safe braking distance: the minimum warning distance is calculated according to the speed of the ego vehicle and the speed of the vehicle in front, and compared with the actual target distance, a safe braking distance collision warning strategy is formulated; (3) Warning strategy based on coupling of TTC and minimum safe braking distance: when the relative speed difference between the ego vehicle and the vehicle in front is small, and when the actual detection distance is less than the safe set distance, the collision warning method based on minimum safe braking distance is prone to trigger a high-level warning, while the actual risk does not require a warning, resulting in a false alarm. The warning strategy based on collision time coupling minimum safe braking distance can effectively improve the accuracy of collision warning;

[0163] The driving status discussion includes: establishing a forward collision warning model for different driving states of the preceding vehicle to meet different traffic operating environments, including a stationary state, a uniform speed state, a uniform deceleration state, a uniform acceleration state and a maximum braking state.

[0164] Early warning scenario analysis specifically includes the following steps:

[0165] S1. Constructing a Gaussian plane rectangular coordinate system: Since the geographic location information received by the vehicle-mounted equipment is in the form of longitude and latitude, it is relatively complicated to calculate the relative position distance between vehicles and construct the vehicle model. Therefore, it is necessary to use the Gauss-Rüger projection technology to convert the initially received longitude and latitude coordinates (P, L) into more intuitive Gaussian plane coordinates (x, y), laying the foundation for the establishment of the subsequent safety collision model. The vehicle's position coordinates (xi, yi) in the Gaussian plane coordinate system can be achieved by the following steps:

[0166]

[0167] Where: x i is the horizontal coordinate of the Gaussian coordinate system, the unit is m; y i is the ordinate of the Gaussian coordinate system, in m; X0 is the arc length of the meridian from the equator to the parallel circle of latitude P, in m; l is the degree difference between the longitude L of the projection point and the meridian longitude L0 of the longitude zone axis, where l = L-L0; L0 = 6n-3, n = round [(L+3) / 6]; N is the radius of curvature of the yoke, in m; t = tan P, ζ = e′cosP;

[0168] Where: X0 = (1-e 2 )(R0P+R1sin2P+R2sin4P+R3sin6P+R4sin8P);

[0169] Where: R0, R1, R2, R3, R4 are parameters, and their specific meanings are as follows:

[0170]

[0171] Where: m = 0.0033528; e = 0.0818192.

[0172] S2. Classification of vehicle collision types: In actual vehicle driving, there is no need to provide collision warning reminders for all vehicles. Therefore, it is necessary to classify vehicles that may collide, and effectively apply corresponding warning decisions and reminder methods for different types of collision types.

[0173] Through the constructed Gaussian plane coordinate system, the coordinates and heading angle information of the vehicle can be accurately obtained. The patent of this invention divides the possible collision types into head-on collision and rear-end collision according to the driving directions of the vehicle and the front vehicle: when the vehicle and the front vehicle are driving towards each other, a head-on collision may occur; when the vehicle and the front vehicle are driving in the same direction, a rear-end collision may occur.

[0174] The present invention sets an angle threshold Δ=10° to better meet the actual collision situation, and the collision judgment condition is:

[0175]

[0176] Where: It is the angle between the line connecting the center coordinates of the vehicle and the front vehicle and the direction of travel of the vehicle (relative heading angle).

[0177] S3. Construct a collision warning strategy based on TTC: Based on the relationship between the relative driving state of the vehicles and the required distance for following braking, a collision time warning model is established. When the time of the impending collision reaches the set warning threshold time, the system will immediately issue a corresponding prompt message.

[0178] The calculation of TTC follows the following formula:

[0179] Where: Δd is the relative distance between the vehicle and the front vehicle, in meters; Δv is the relative speed of the two vehicles, in meters per second; v n is the vehicle speed, in m / s; v m is the speed of the vehicle in front, in m / s. The risk of a collision between the two vehicles is inversely proportional to the TTC value. The smaller the TTC value, the greater the risk of a collision. This collision warning strategy can show high reliability when the vehicle is traveling at a low speed.

[0180] According to the requirements for setting the collision warning system, the prompt time should be greater than the reaction time. Therefore, the forward collision warning strategy level is set as shown in Table 1 in the example of the present invention:

[0181] Table 1

[0182]

[0183] The values ​​in Table 1 are threshold times. Warnings are issued by judging fixed threshold times to ensure the safety of the driving process.

[0184] S4. Warning strategy based on minimum safe braking distance: Calculate the minimum warning distance based on the speed of the vehicle and the speed of the vehicle in front, and compare it with the actual target distance, and formulate a safe braking distance collision warning strategy based on this. In the vehicle's braking process, the safe collision avoidance time includes the driver's reaction time t1 from identifying the danger to starting braking, the coordination time t2 from the start of braking with the brake pedal to the braking effect (take t2 = 0.2 seconds), the growth time t3 from the braking effect to the hydraulic brake generating braking action, and the duration t4 without braking force. The specific calculation formula is as follows:

[0185] Assuming the initial speed of the vehicle is v0, the distance d1 traveled in the time period t1 is: d1 = v0t1;

[0186] The travel distance d2 during the time period t2 is: Where: amax is the maximum deceleration;

[0187] The travel distance d3 during the time period t3 is:

[0188] In summary, the total braking distance D of the vehicle is: Based on actual safety considerations, during the braking process of a vehicle, t1 = 1.5s, t2 = 0.2s, and t3 = 0.3s are generally taken.

[0189] The calculation formula for the duration without braking force t4 is: Where v is the vehicle speed in m / s; g is the acceleration due to gravity, g=9.8 in m / s 2 , λ is the road adhesion coefficient. This paper only considers dry roads and takes λ = 0.75; where the maximum deceleration a max =λg.

[0190] Specifically, the total braking distance D of the vehicle is:

[0191]

[0192] In summary, the warning strategies adopted are as follows: (1) When the relative distance between the ego vehicle and the vehicle in front is greater than the minimum safe braking distance, the collision warning is not triggered; (2) When the relative distance between the ego vehicle and the vehicle in front is less than or equal to the minimum safe braking distance, the collision warning is activated.

[0193] Example 3

[0194] Combination Figures 1 to 8 , based on the coupling warning strategy of TTC and minimum safe braking distance: When the relative speed difference between the vehicle and the front vehicle is small, and when the actual detection distance is less than the safe set distance, the collision warning method based on the minimum safe braking distance is likely to trigger a high-level warning, while the actual risk does not require a warning, resulting in a false alarm. The warning strategy based on the collision time coupling minimum safe braking distance can significantly improve the accuracy of the collision warning:

[0195] According to the above formula, the collision warning calculation formula based on the coupling of collision time TTC and minimum safe braking distance is as follows:

[0196] The specific steps of the fusion collision warning process are as follows: (1) obtain the vehicle's safe braking distance, relative collision time, specific location information of the target, etc. (2) determine the relationship between the relative target distance between vehicles and the minimum safe braking distance; (3) calculate the TTC warning time value and compare it with the set threshold time to make a level judgment; (4) according to the determined collision risk warning level, issue corresponding warning and prompt information.

[0197] Example 4

[0198] The driving status discussion specifically includes the following steps:

[0199] A forward collision warning model is established for different driving states of the preceding vehicle, and the collision time TTC value is calculated to meet different traffic operation environments, including stationary state, uniform speed state, uniform deceleration state, uniform acceleration state and maximum braking state. The calculation of each state is as follows:

[0200] When the vehicle is stationary, v m =0, then:

[0201] When the current vehicle is at a constant speed, v m >0, a m =0, then:

[0202] When the current vehicle is in a uniform deceleration state, m g<a m <0, then:

[0203] When the current vehicle is in a uniform acceleration state, a m >0, then:

[0204] When the vehicle is in the maximum braking state, a m =λ m g, then:

[0205] In summary, the collision warning strategy classification relationship based on the coupling of TTC and minimum safe braking distance is as follows:

[0206] Very dangerous level relationship:

[0207] Moderate hazard level relationship: or

[0208] General hazard level relationship:

[0209] Mild hazard level relationship:

[0210] Safety and no-collision risk level relationship:

[0211] Where: D is the target distance between the vehicle and the front vehicle.

[0212] The steps of the fusion collision warning process are as follows: Figure 3 As shown in the figure: (1) obtain the vehicle's safe braking distance, relative collision time, specific location information of the target, etc.; (2) determine the relationship between the relative target distance between vehicles and the minimum safe braking distance; (3) calculate the TTC warning time value and compare it with the set threshold time to make a level judgment; (4) according to the determined collision risk warning level, issue corresponding warning and prompt information.

[0213] Example 5

[0214] This embodiment provides a multi-level automatic driving forward collision safety takeover method, such as Figure 4 As shown, it includes: a perception information acquisition unit, an emergency state judgment unit, a system warning prompt unit and a driving operation control unit; the perception information acquisition unit 001, by integrating the roadside unit and the vehicle-mounted sensor, collects various data of vehicle driving in real time, including vehicle speed, position, acceleration and status information of the surrounding environment; the emergency state judgment unit 002, estimates the vehicle driving time based on the perception information data, and further calculates the remaining time for automatic driving judgment, and confirms the reminder type of the human-vehicle takeover warning; the warning result judgment unit 003, based on the relationship between the remaining driving time and the set warning time threshold, generates different categories of system warning judgment results; the takeover information prompt unit 004, according to the different levels of the warning result, sends a corresponding takeover prompt to the driver.

[0215] Depending on the severity of the warning, the intensity of the prompt will also be different; the driving operation control unit 005, after the takeover warning is triggered, is responsible for achieving a smooth transition between the automatic driving mode and the manual control mode, and making corresponding driving state adjustments.

[0216] For vehicles of different levels of autonomous driving, the driver's attention level will vary during road driving depending on the level of autonomous driving. When a situation arises where the driver needs to take over the current driving task, the urgency of the takeover warning will also vary. Therefore, the system needs to provide the driver with different levels and combinations of warning prompts based on the urgency of the takeover requirement.

[0217] According to the judgment, calculation and comparison of the set warning time threshold, the system can generate different types of warning prompts, thereby effectively improving the safety of autonomous driving. The embodiment of the present invention sets the corresponding warning time threshold, and performs emergency processing according to different system warning judgment results, and provides warning prompt methods of different degrees and combinations to ensure the best prompt effect.

[0218] The emergency state judgment unit 002 is used to calculate the remaining time of automatic driving, and compare it with the set warning time threshold to generate a system warning judgment result. The warning time threshold includes a first warning time threshold, a second warning time threshold, a third warning time threshold and a fourth warning time threshold; the system warning judgment result includes no system warning result, a first type of system warning judgment result, a second type of system warning judgment result, a third type of system warning judgment result and a fourth type of system warning judgment result. According to an embodiment of the present invention, the first warning time threshold is 5 minutes, the second warning time threshold is 3 minutes, the third warning time threshold is 2 minutes, and the fourth warning time threshold is 1 minute. Other warning time thresholds can also be set according to actual needs.

[0219] The warning result judgment unit 003 is used to provide the driver with takeover prompt information of different urgency levels based on the comparison between the driver's remaining time and the set warning time threshold. If the vehicle judges that the remaining time is greater than the first warning time threshold, there is no system warning result; if the remaining time is not greater than the first warning time threshold but greater than the second warning time threshold, it is a first-type system warning judgment result; if the remaining time is not greater than the second warning time threshold but greater than the third warning time threshold, it is a second-type system warning judgment result; if the remaining time is not greater than the third warning time threshold but greater than the fourth warning time threshold, it is a third-type system warning judgment result; if the remaining time is not greater than the fourth warning time threshold, it is a fourth-type system warning judgment result;

[0220] The takeover information prompt unit 004 generates takeover prompt information of corresponding urgency according to the system warning judgment result, including: no takeover prompt information, first type of takeover prompt information, second type of takeover prompt information, third type of takeover prompt information and fourth type of takeover prompt information.

[0221] When the vehicle determines that the remaining time is greater than the first warning time threshold, there is no system warning result. In response to the no takeover prompt information, the remaining time for the current vehicle's automatic driving is sufficient, and there is no need to take over control of the vehicle, and the driver is in a normal driving state.

[0222] When the vehicle determines that the remaining time is not greater than the first warning time threshold, but greater than the second warning time threshold, the first type of system warning judgment result appears, and the first type of takeover prompt information is responded to. At this time, the driver is reminded that there is not much time left for automatic driving, and manual takeover control is required, and there is still a certain buffer time.

[0223] When the vehicle judges that the remaining time is not greater than the second warning time threshold, but greater than the third warning time threshold, the second type of system warning judgment result appears, and the second type of takeover prompt information is responded to. At this time, the remaining time of the vehicle's automatic driving is short, and the driver needs to be further prompted to take over the vehicle control. When the vehicle judges that the remaining time is not greater than the third warning time threshold, but greater than the fourth warning time threshold, the third type of system warning judgment result appears, and the third type of takeover prompt information is responded to. At this time, the remaining time of the vehicle's automatic driving is extremely short, and the driver needs to be prompted to take over the vehicle control as soon as possible. When the vehicle judges that the remaining time is not greater than the fourth warning time threshold, the fourth type of system warning judgment result appears, and the fourth type of takeover prompt information is responded to. At this time, the remaining time of the vehicle's automatic driving is about to run out, and the collision warning level has reached a very dangerous level. At this time, the driver is prompted to immediately enter manual takeover of vehicle control.

[0224] Example 6

[0225] Furthermore, the takeover information prompting unit 004 based on visual-auditory-tactile interaction also includes: a dashboard LED light strip 1, a speaker 2, and a tactile interaction hardware 3. The takeover information prompting unit 004 controls the dashboard LED light strip 1, the speaker 2, and the tactile interaction hardware 3 according to the received takeover reminder information, and emits light strips of different colors and different flashing frequencies and sounds of different categories and different vibration frequencies to prompt the driver.

[0226] Specifically, based on the system warning judgment result, takeover prompt information of the corresponding emergency level is generated. According to the urgency of the takeover warning, the prompt module is controlled to emit light strips of different colors and flashing frequencies and sounds of different categories and vibration frequencies to provide multimodal prompts to the driver, thereby ensuring that the driver can respond in time and take corresponding operations in different emergency situations.

[0227] When the vehicle determines that the remaining time is greater than the first warning time threshold, there is no system warning result, then in response to the no takeover prompt information, the system warning prompt has no color and flashing frequency changes of the instrument panel LED light strip 1, no alarm sound and vibration changes, and the driver is in a normal driving state;

[0228] When the vehicle determines that the remaining time is not greater than the first warning time threshold, but greater than the second warning time threshold, the remaining time for autonomous driving is sufficient, and the first type of system warning judgment result appears, then in response to the first type of takeover prompt information, the system warning prompt is that the dashboard LED light strip 1 emits a relatively gentle blue light with a flashing frequency of 0-1 times / second, and there is no need for the speaker 2 and tactile interaction hardware 3 to perform sound warning and vibration warning. At this time, the driver is reminded to manually take over the control, and there is still a certain buffer time;

[0229] When the vehicle determines that the remaining time is not greater than the second warning time threshold but greater than the third warning time threshold, the remaining time for autonomous driving is sufficient, and the second type of system warning judgment result appears, then in response to the second type of takeover prompt information, the system warning prompt is that the instrument panel LED light strip 1 emits a gentle green light with a flashing frequency of 0-1 times / second, the speaker 2 emits a female voice-type prompt sound, including words such as "attention", and the tactile interaction hardware 3 vibrates at a frequency f<10Hz;

[0230] When the vehicle determines that the remaining time is not greater than the third warning time threshold but greater than the fourth warning time threshold, the remaining time for automatic driving is very short, and the third type of system warning judgment result appears, then in response to the third type of takeover prompt information, the system warning prompt is that the instrument panel LED light strip 1 emits a more conspicuous orange light with a flashing frequency of 1-2 times / second, the speaker 2 emits a male alarm type prompt sound, including words such as "beep" and "beep", and the vibration frequency of the tactile interaction hardware 3 is 10Hz≤f≤200Hz;

[0231] When the vehicle determines that the remaining time is not greater than the fourth warning time threshold, the remaining time for autonomous driving is about to run out, and the fourth type of system warning judgment result appears, then in response to the fourth type of takeover prompt information, the system warning prompt is that the dashboard LED light strip 1 emits a conspicuous red light with a flashing frequency of 2-4 times / second, the speaker 2 emits a male alarm type prompt sound, including words such as "danger", and the tactile interaction hardware 3 vibrates at a frequency of f>200Hz. At this time, the driver is reminded that the autonomous driving is about to end and needs to enter manual takeover control immediately;

[0232] The driving operation control unit 005 is used to complete the conversion of the manual control mode according to the takeover warning information, prompt the driver to complete the vehicle takeover, and make corresponding driving state adjustments. At the same time, during the automatic driving process, the driver's driving mode is monitored in real time to ensure that it is in the manual driving control mode or the automatic driving control mode, thereby improving driving safety.

[0233] When the fourth warning time threshold is about to expire, it is determined that the driver has not completed the control of the vehicle. At this time, the vehicle quickly enters an emergency state, and the automatic driving safety warning system controls the vehicle to decelerate and enter the emergency mode, turning on the emergency lights. When the relative heading angle When the left turn signal is turned on, the vehicle control system controls the vehicle to change lanes to the left and slow down, and the left turning angle is Δ; when the relative heading angle When the right turn signal is turned on, the vehicle control system controls the vehicle to change lanes to the right and slow down, and the right turning angle is Δ. The control process is completed.

[0234] It should be noted that the control prompt module can emit instrument panel light strips of different colors and flashing frequencies according to the urgency of the takeover warning in the example of the present invention, and multiple colors can be set, not limited to the blue, orange, green and red mentioned in this embodiment. The sound warning is set with female voice type and male alarm type, and only fixed combinations of different timbres and volumes can be set to distinguish different urgency levels; the tactile prompt setting can only select different frequencies.

[0235] In the example of the present invention, the first warning time threshold, the second warning time threshold, the third warning time threshold and the fourth warning time threshold are set to divide the automatic driving takeover demand into four levels. The four levels of dashboard LED light strip warning effects correspond to four different colors of lights: blue (prompt), green (reminder), orange (reminder) and red (warning), and emit sounds of different flashing frequencies or different categories and vibration frequencies for warning prompts.

[0236] According to the relationship between the remaining time of autonomous driving and the set warning time threshold, system warning judgment results of different urgency levels are generated, thereby providing corresponding prompt effects for the warning results, so that the driver can intuitively and quickly understand the urgency of taking over the warning.

[0237] A multi-level human-vehicle takeover warning prompt method based on visual-auditory-tactile interaction mentioned in the example of the present invention is shown in Table 2 and Table 3 below:

[0238] Table 2

[0239]

[0240] Table 3

[0241]

[0242] Example 7

[0243] The multi-level safety warning system and method for autonomous driving vehicle takeover based on visual-auditory-tactile interaction proposed in this example can clearly and effectively send takeover information to the driver when the vehicle encounters an emergency or exceeds the system operation area, so that the driver can respond to the takeover in a timely and efficient manner. The visual takeover interaction is to transmit the takeover request, system emergency status and directional operation information to the driver by changing the color, frequency and rotation direction of the LED light strip on the dashboard.

[0244] Auditory takeover interaction can eliminate the impact of takeover information on the driver's ability to restore situational awareness through vision, including designing two takeover reminder methods: voice type and alarm type. Male and female voices are connected to the speakers to express specific prompt information through voice. The voice type takeover is set to a female prompt voice, mainly including words such as "attention", and the alarm type is set to a male prompt voice, mainly including "danger" and short "beep" and "beep" sounds.

[0245] Tactile takeover interaction uses vibration tactile stimulation as a takeover signal to enable the driver to effectively receive the takeover request. The tactile interaction design is located in the tactile interaction hardware of vehicle components such as seat cushions, backrests and steering wheels. It transmits takeover information by stimulating different parts of the driver's body and generates vibrations in a specific direction on the steering wheel to provide the driver with directional operation information as a prompt aid.

[0246] The combination of multiple interaction modes is based on the visual-auditory-tactile interaction mode combination, which can combine the advantages of various interaction types, so that the driver can react more quickly and ensure driving safety. Different levels of reminder combinations are set in different emergency situations to assist the driver in judging the degree of urgency and making corresponding explanation responses.

Claims

1. A multi-level autonomous driving forward collision safety warning system and method, characterized in that: It includes a data information collection unit, a collision warning calculation unit and a human-computer interaction reminder unit; After the data information acquisition unit collects data, the collision warning calculation unit performs data information fusion processing on the corresponding data. The human-computer interaction reminder unit receives the instruction from the collision warning calculation unit and issues the corresponding level of warning information according to the set warning degree and level definition. When receiving the alarm information from the vehicle, the driver is reminded to take measures to avoid collision. If the collision risk is eliminated, the warning prompt is cancelled. If the driver is in poor condition and cannot take over the vehicle and take braking measures, or the driver does not take over the vehicle, the automatic driving system will automatically enter the minimum risk control mode, gradually reduce the speed, and ensure that the vehicle brakes to a stop before a collision occurs; The data information fusion processing is to fuse the data collected by the data information collection unit, and the content of the fusion processing includes output layer, modal feature extraction, and feature fusion; The input layer inputs the video data and radar data into the model. The video data uses a deep learning network to extract image features, and the radar data uses a Chinese BEV conversion network to extract point cloud features. The modal feature extraction includes video modal feature extraction and radar modal feature extraction; video modal feature extraction uses FasterR-CNN to extract the spatial features of the target in the video and outputs a two-dimensional feature map; radar modal feature extraction uses PointNet++ to extract local and global geometric features and output a two-dimensional or three-dimensional feature tensor. The output content is: F image =f CNN (I); F radar =f PointNet (P); Among them, F image represents the features extracted from the video data, F rader represents the features extracted from radar data, f CNN It means that the convolutional neural network extracts high-dimensional features of the image through convolutional layers, pooling layers and nonlinear activation functions, I represents the image data in the video data, and f PointNet represents a network for processing point cloud data, and P represents radar point cloud data; The feature fusion described above requires aligning the features of different modalities into a unified reference coordinate system through a modality alignment mechanism; Radar data is projected from the radar coordinate system to the image plane through the camera calibration matrix to establish the spatial relationship between the radar coordinate system and the camera coordinate system; The image data in the video data uses stereo vision or structured light technology to obtain the depth information in the video. Each pixel in the image data can be mapped to a three-dimensional space to obtain the depth value of the pixel. These depth information converts the video data from a two-dimensional image to a point cloud in a three-dimensional space, so that the video data and radar data are located in the same three-dimensional coordinate space for subsequent feature fusion. Finally, the feature tensor is convolved point by point to achieve deep fusion. The final fusion feature is as follows: F fused =f CNN ([F image ,F radar ]); Among them, F fused represents the final fusion feature after being processed by the CNN network, [F image ,F rader ] means concatenating the video and radar features together to form a new feature vector.

2. A multi-level autonomous driving forward collision safety warning system and method according to claim 1, characterized in that: The data information acquisition unit uses a speed sensor to collect relevant speed data such as the speed and acceleration of the vehicle, and uses a camera and millimeter radar waves to collect the speed, position and azimuth angle data of the forward vehicle; The collision warning calculation unit performs data information fusion processing on the data information collection unit, and combines the driving state information of the vehicle to execute a certain collision warning algorithm to calculate and evaluate whether there is a potential collision safety risk. If it is determined that there is a risk, a warning instruction is issued to the human-computer interaction reminder unit; After point-by-point convolution operation, deep fusion is achieved. Deep fusion includes early fusion and late fusion. Early fusion is to splice the feature maps of video images and radar data together at the initial stage of data, so that video features and radar features are used as parallel inputs and passed into a shared convolution layer to merge the feature maps of the two modalities into a large feature map. Late fusion is that the features of video and radar data are processed by their own convolution layers respectively, so that video and radar data can learn features independently in different channels, and then fuse them through specific operations. The deeply fused feature map is further processed through several convolutional layers to gradually extract more advanced and abstract features. During the feature extraction process, the convolutional neural network automatically learns the features and reduces redundant information. The driver status is that the automatic driving system monitors the driver's driving behavior at all times and determines whether the driver has the ability to take over the vehicle by monitoring the driver's blinking frequency in the last 10 seconds.

3. The multi-level automatic driving forward collision safety warning system and method as claimed in claim 1, characterized in that: A forward collision warning decision algorithm based on collision time and minimum safe braking distance is constructed in the collision warning calculation unit, where the collision time is TTC; The forward collision warning decision algorithm includes warning scenario analysis, warning strategy design and driving status discussion; The warning scenario analysis includes: A1. Establishing Gaussian plane coordinate system: transform the latitude and longitude information of the vehicle and the preceding vehicle into position coordinates in the Gaussian plane coordinate system to obtain the center coordinates of the vehicle, which is convenient for subsequent calculation of vehicle safety collision warning; A2. Vehicle collision type: In actual driving, there is no need to issue collision warnings to all vehicles. By classifying vehicles that may collide, corresponding warning decisions and reminder methods can be effectively applied to different collision types. Through the Gaussian plane coordinate system, the vehicle coordinates and heading angle information can be accurately obtained, and the possible collision types can be divided into forward collision and rear-end collision. The early warning strategy design includes: B1. Collision strategy based on TTC: Based on the analysis of the relative speed between the vehicle and the preceding vehicle, the relative collision event is calculated, and the warning time threshold is set to perform warning classification processing on the collision risk; the collision time TTC is a measure of the time required for a collision between the vehicle and the obstacle; B2. Warning strategy based on minimum safe braking distance: Calculate the minimum warning distance based on the speed of the vehicle and the speed of the preceding vehicle, and compare it with the actual target distance to formulate a safe braking distance collision warning strategy; B3. Coupling warning strategy based on TTC and minimum safe braking distance: When the relative speed difference between the vehicle and the preceding vehicle is small and the actual detection distance is less than the safety set distance, the collision warning method based on the minimum safe braking distance is likely to trigger a high-level warning, while the actual risk does not require a warning, resulting in a false alarm. The warning strategy based on the collision time coupled with the minimum safe braking distance can effectively improve the accuracy of the collision warning. The driving status discussion includes: establishing a forward collision warning model for different driving states of the preceding vehicle to meet different traffic operating environments, including a stationary state, a uniform speed state, a uniform deceleration state, a uniform acceleration state and a maximum braking state.

4. The multi-level autonomous driving forward collision safety warning system and method according to claim 3, characterized in that: The analysis process of the warning scenario specifically includes the following steps: SA1. Constructing Gaussian plane rectangular coordinate system: When calculating the relative position distance between vehicles and constructing the vehicle model, the Gauss-Rüger projection technology is required to convert the initially received latitude and longitude coordinates (P, L) into more intuitive Gaussian plane coordinates (x, y), laying the foundation for the establishment of the subsequent safety collision model. The vehicle position coordinates (x i ,y i ) can be achieved by following the steps below: Where: x i is the abscissa of the Gaussian coordinate system; y i is the ordinate of the Gaussian coordinate system; X0 is the length of the meridian arc from the equator to the parallel circle of latitude D; l is the difference in degrees between the longitude L of the projection point and the meridian longitude L0 of the longitude zone axis; The calculation method of the data in the formula is: l = L-L0; L0 = 6n-3, n = round [(L+3) / 6]; N is the radius of curvature of the yoke; t = tan D, ζ = e'cos D; Where: X0 = (1-e 2 )(R0D+R1sin2D+R2sin4D+R3sin6D+R4sin8D); Where: R0, R1, R2, R3, R4 are parameters, and their specific meanings are as follows: Where: m = 0.0033528; e = 0.0818192; SA2. Obtaining vehicle collision types: Classify vehicles that may collide, and accurately obtain the coordinates and heading angle information of the vehicles through the constructed Gaussian plane coordinate system. According to the driving directions of the self-vehicle and the front vehicle, the possible collision types are divided into forward collision and rear-end collision. When the self-vehicle and the front vehicle are driving towards each other, there may be a forward collision; when the self-vehicle and the front vehicle are driving in the same direction, there may be a rear-end collision. The collision judgment condition is: In the formula, It is the angle between the line connecting the center coordinates of the vehicle and the front vehicle and the driving direction of the vehicle. The angle is the relative heading angle; the angle threshold Δ=10°.

5. The multi-level autonomous driving forward collision safety warning system and method according to claim 3, characterized in that: The design of the early warning strategy includes the following steps: SB1. Construct a collision warning strategy based on TTC: Based on the relationship between the relative driving status of the vehicles and the distance required for following braking, a collision time warning model is established; when the time to an impending collision reaches the preset warning time threshold, the system will immediately issue a corresponding prompt message; the calculation of TTC follows the following formula: Where: Δd is the relative distance between the vehicle and the front vehicle; Δv is the relative speed of the two vehicles; v n is the vehicle speed; v m is the speed of the vehicle in front; the risk of a collision between the two vehicles is inversely proportional to the TTC value. The smaller the TTC value, the greater the risk of a collision. This collision warning strategy can show high reliability when the vehicle is traveling at a low speed; SB2, warning strategy based on minimum safe braking distance: Calculate the minimum warning distance based on the speed of the vehicle and the speed of the preceding vehicle, and compare it with the actual target distance, based on which a safe braking distance collision warning strategy is formulated; In the vehicle's braking process, the safe collision avoidance time includes the driver's reaction time t1 from identifying danger to starting braking; the coordination time t2 from the start of braking with the brake pedal to the braking effect, t2 is set to 0.2 seconds; the growth time t3 from the braking effect to the hydraulic brake generating braking action, and the duration of no braking force t4; The specific calculation formula is as follows: Assuming the initial speed of the vehicle is v0, the distance d1 traveled during the time period t1 is: d1=v0t1; The travel distance d2 during the time period t2 is: In the formula, a max is the maximum deceleration, m / s 2 ; The travel distance d3 during the time period t3 is: The total braking distance D of the vehicle is obtained as: During the braking process of the vehicle, the calculation formula for the duration t4 without braking force is: In the formula, v is the vehicle speed; g is the acceleration of gravity, λ is the road adhesion coefficient, and the maximum deceleration a is max =λg; Specifically, the total braking distance D of the vehicle is: SB3. Coupling warning strategy based on TTC and minimum safe braking distance: The collision warning calculation formula based on the coupling of collision time TTC and minimum safe braking distance is as follows:

6. The multi-level automatic driving forward collision safety warning system and method according to claim 3, characterized in that: The driving status discussion specifically includes the following steps: SC1. Build a corresponding forward collision warning model for various driving states that the vehicle ahead may present, including stationary, constant speed, constant deceleration, constant acceleration and maximum braking state, and calculate the collision time TTC value under each state to ensure that the system can adapt to different traffic operating environments; SC2. Calculate the TTC value in each state: When the vehicle is stationary, v m =0, then: When the current vehicle is at a constant speed, v m >0, a m =0, then: When the current vehicle is in a uniform deceleration state, m g<a m <0, then: When the current vehicle is in a uniform acceleration state, a m >0, then: When the vehicle is in the maximum braking state, a m =λ m g, then:

7. The multi-level automatic driving forward collision safety warning system and method according to claim 4, characterized in that: The autonomous driving includes a safety warning system for human-vehicle takeover, which includes a perception information acquisition unit, an emergency state judgment unit, a warning result judgment unit, a takeover information prompt unit, and a driving operation control unit; The perception information acquisition unit integrates the roadside unit and the vehicle-mounted sensors to collect the vehicle's speed, position, acceleration and surrounding environment status information in real time; The emergency state judgment unit estimates the vehicle driving time based on the perception information data, calculates the remaining time for automatic driving judgment, and confirms the reminder type of the human-vehicle takeover warning; The warning result judgment unit generates different types of system warning judgment results based on the relationship between the remaining driving time and the set warning time threshold; The takeover information prompt unit sends corresponding takeover prompts to the driver according to the different levels of the warning results. The intensity of the prompt method will also vary according to the severity of the warning. After the takeover warning is triggered, the driving operation control unit is responsible for achieving a smooth transition between the automatic driving mode and the manual control mode and making corresponding driving status adjustments.

8. The multi-level automatic driving forward collision safety warning system and method according to claim 7, characterized in that: The emergency state judgment unit specifically includes: Warning time thresholds, the warning time thresholds include a first warning time threshold, a second warning time threshold, a third warning time threshold and a fourth warning time threshold, the first warning time threshold is greater than the second warning time threshold, the second warning time threshold is greater than the third warning time threshold, and the third warning time threshold is greater than the fourth warning time threshold; System warning judgment results, the system warning judgment results include no system warning results, the first type of system warning judgment results, the second type of system warning judgment results, the third type of system warning judgment results and the fourth type of system warning judgment results.

9. The multi-level automatic driving forward collision safety warning system and method according to claim 8, characterized in that: When the vehicle determines the remaining time, the relationship between the remaining time and the warning time threshold is: When the vehicle determines that the remaining time is greater than the first warning time threshold, the no system warning result occurs; When the vehicle judged remaining time is less than or equal to the first warning time threshold but greater than the second warning time threshold, the first type of system warning judgment result appears; When the vehicle judges that the remaining time is less than or equal to the second warning time threshold but greater than the third warning time threshold, the second type of system warning judgment result appears; When the vehicle judged remaining time is less than or equal to the third warning time threshold but greater than the fourth warning time threshold, the third type of system warning judgment result appears; When the vehicle judged remaining time is less than or equal to the fourth warning time threshold, the fourth type of system warning judgment result appears.

10. The multi-level automatic driving forward collision safety warning system and method according to claim 8, characterized in that: The takeover warning prompt modes include takeover reminder based on visual interaction, takeover reminder based on auditory interaction and takeover reminder based on tactile interaction; The types of warning information provided by the early warning include: When responding to the no-takeover prompt message, the system will warn that there is no change in the color or flashing frequency of the dashboard LED light, and there is no change in the alarm sound or vibration; When responding to the first type of takeover prompt information, the system warning prompt includes a blue LED light strip on the instrument panel, a flashing frequency of 0-1 times / second, and no alarm sound or vibration frequency change; When responding to the second type of takeover prompt information, the system warning prompt includes a green LED light strip on the instrument panel, a flashing frequency of 0-1 times / second, a voice alarm sound, and a vibration frequency f<10Hz; When responding to the third type of takeover prompt information, the system warning prompt includes an orange LED light strip on the instrument panel, a flashing frequency of 1-2 times / second, a voice alarm sound, and a vibration frequency of 10Hz≤f≤200Hz; When responding to the fourth type of takeover prompt information, the system warning prompt includes a red LED light strip on the instrument panel, a flashing frequency of 2-4 times / second, an alarm type alarm sound, and a vibration frequency f>200Hz; When the fourth warning time threshold is about to expire and the driver has not taken over the vehicle: relative heading angle Turn on the left turn signal, and the vehicle control system controls the vehicle to change lanes to the left and slow down, with a left turning angle of Δ; When the relative heading angle , turn on the right turn signal, and the vehicle control system controls the vehicle to change lanes to the right and slow down, with a right turning angle of Δ.

Images