V2X-based pedestrian crossing and two-wheeled vehicle early warning system and early warning method thereof

Abstract

The invention discloses a V2X-based pedestrian and two-wheeled vehicle crossing early warning system and an early warning method thereof.An environment sensing module is arranged on a vehicle and used for collecting obstacle information Za around the vehicle and outputting fused obstacle information Za1; the vehicle-mounted communication module receives the obstacle information Za1 output by the vehicle environment sensing module, broadcasts and sends the real-time obstacle information Za1 detected by the vehicle through a V2X channel and receives the real-time obstacle information Za2 of other vehicles; the information processing module obtains obstacle information Za1 from a vehicle and obstacle information Za2 from V2X and judges the obstacle information Za1 and the obstacle information Za2 to obtain obstacle information Za3 fused with V2X information; and the vehicle body control module controls the vehicle according to the obstacle information Za3 fused with the V2X information. The pedestrian and two-wheeled vehicle crossing early warning system based on V2X and the early warning method thereof have the advantages that obstacle targets in a visual blind area can be detected in scenes such as a'ghost probe ', beyond-visual-range perception is achieved, and the traffic accident rate can be reduced.

Description

Technical Field

[0001] The present invention relates to a traffic safety warning technology, in particular to a V2X-based warning system for crossing pedestrians and two-wheeled vehicles and a warning method thereof. Background Art

[0002] In recent years, with the development of social economy and the improvement of people's living standards, the number of cars owned by residents has continued to increase. While providing traffic convenience, vehicles have also greatly increased the probability of traffic accidents. Among them, the "ghost head" incident, in which pedestrians or two-wheeled vehicles move in the blind spot of the motor vehicle driver's vision, causing the driver to fail to brake in time and causing traffic accidents, has become one of the main traffic accidents causing casualties. "Ghost head" generally refers to other vehicles or obstacles blocking the line of sight on the left and right sides of the front of the moving vehicle, that is, there are blind spots on the left and right sides of the driver's front, and non-motor vehicles or pedestrians suddenly jump out of the blind spots; in this case, the driver fails to avoid in time, often resulting in tragic car injuries and deaths. Because the incident happened suddenly, neither side had time and space to react and dodge, which caught people off guard, so it is called "ghost head".

[0003] There are two main types of solutions to the "ghost peeking" phenomenon of pedestrians and two-wheeled vehicles crossing the road in the existing technology: one is to use the vehicle's sensors such as cameras or radars to identify and judge the possible pedestrians or two-wheeled vehicles, and then issue an early warning to the vehicle; the other is to use roadside units to monitor pedestrians or two-wheeled vehicles on the road, and then send them to vehicles on the road in the form of broadcasts.

[0004] However, in the specific implementation of the above two methods, if only the first type of single-vehicle sensor is used for identification and judgment, there are problems such as being easily affected by weather or the vehicle's perception field of view, slow response speed, low accuracy, and weak anti-interference ability; if only the second type of relying on roadside units to monitor pedestrians or two-wheeled vehicles is used, there are problems such as high investment cost and signal redundancy. Summary of the invention

[0005] The present invention is to avoid the deficiencies existing in the above-mentioned prior art and provide a V2X-based warning system for pedestrians and two-wheeled vehicles crossing the road and a warning method thereof, so as to accurately warn of the "ghosting" phenomenon in road traffic and remind the driver, effectively avoid traffic accidents, and improve the safety and reliability of road traffic.

[0006] The present invention adopts the following technical solutions to solve the technical problems.

[0007] A V2X-based warning system for pedestrians and two-wheeled vehicles crossing the road in the present invention includes an environment perception module, a vehicle communication module, an information processing module and a body control module;

[0008] The environment perception module is arranged on the vehicle, and the multiple sensors of the environment perception module are used to collect multiple obstacle information Za around the vehicle; the obstacle information Za1 is obtained after a fusion process is performed on the multiple obstacle information Za around the location of the vehicle, and then the obstacle information Za1 is sent to the vehicle communication module and the information processing module;

[0009] The vehicle-mounted communication module receives the obstacle information Za1 sent by the vehicle's environmental perception module, broadcasts the obstacle information Za1 detected by the vehicle to other nearby vehicles through V2X, receives the obstacle information Za2 sent in real time by the vehicle-mounted communication modules of other vehicles with the early warning system near the vehicle, and sends the obstacle information Za2 to the information processing module;

[0010] The information processing module is used to receive the obstacle information Za1 of the vehicle sent by the environment perception module of the vehicle and the obstacle information Za2 received by the vehicle communication module, and perform secondary fusion analysis on the obstacle information Za1 and the obstacle information Za2, and obtain the obstacle information Za3 after the secondary fusion analysis, and then send the obstacle information Za3 to the body control module;

[0011] The vehicle body control module uses the obstacle information Za3 determined by the information processing module and controls the vehicle according to the determined obstacle information Za3.

[0012] The V2X-based warning system for pedestrians and two-wheeled vehicles crossing the road and the warning method thereof are also characterized in that:

[0013] Furthermore, the environment perception module includes a plurality of sensors for collecting obstacle information Za around the vehicle body, and each sensor is used for collecting a type of obstacle information Za.

[0014] Furthermore, the sensor includes but is not limited to a camera, a millimeter wave radar, an ultrasonic radar and a lidar.

[0015] Furthermore, the environmental perception module performs a fusion process on multiple obstacle information Za around the vehicle as follows:

[0016] Step 01: Obstacle position velocity vector ObVe of different obstacle information Za measured by the environment perception module; the obstacle position velocity vector ObVe includes the position coordinate ObX of the obstacle in the X-axis direction, the position coordinate ObY of the obstacle in the Y-axis direction, the velocity value ObVx of the obstacle in the X-axis direction, and the velocity value ObVy of the obstacle in the Y-axis direction; that is: ObVe = [ObX, ObY, ObVx, ObVy]; wherein the X-axis and Y-axis are the XY axes of the ego-vehicle coordinate system;

[0017] Step 02: The environment perception module includes multiple sensors, and the perception result output by each sensor is the position and velocity vector ObVe of m obstacles; calculate the cosine similarity S between any two position and velocity vectors ObVe from different sensors;

[0018] Step 03: When different sensors sense the same obstacle, the i-th sensor is assigned a weight value W i , the confidence of the i-th sensor is Z i ;According to the matching result of the obstacle information in step 02, obtain the sensor sequence that perceives the obstacle and calculate the comprehensive score value, which is the confidence Z of the obstacle. Z Confidence Z Z The calculation formula is as follows: Formula (12);

[0019]

[0020] In formula (12), n is the number of sensors successfully matched in step 02, and i represents the i-th sensor;

[0021] Step 04: Obstacle confidence Z obtained in step 03 Z , select the fused obstacles based on the confidence threshold, and filter out the obstacles with confidence Z Z After the obstacle is too low, the reference obstacle Ob is output i ;

[0022] Step 05: Filter the obstacle types, remove obstacles that are not pedestrians or two-wheeled vehicles, and use them as obstacle information Za1 output by the environment perception module. The obstacle position velocity vector of the obstacle information Za1 is ObVe = [ObX, ObY, ObVx, ObVy].

[0023] Step 06: According to the obstacle information Za1 obtained in step 05, additional information of the obstacle is added, including the absolute time Zt of finding the obstacle, the obstacle type Zl, the confidence level Zz of the obstacle information, and the position and speed information ObVe of the obstacle;

[0024] Furthermore, the information processing module can perform secondary fusion analysis on the obstacle information Za1 sensed by the vehicle and the obstacle information Za2 received from other vehicles with the system, and use the obstacle information Za3 after the secondary fusion analysis as the input of the body control module.

[0025] Furthermore, the information processing module will sort the obstacles according to the collision time TTC (Time to Collision) of the perceived and received obstacle information, and select the obstacle with the smallest TTC as the input of the body control module. If there are multiple obstacles with the same TTC, the obstacle closest to the vehicle position is selected as the input of the body control module.

[0026] The specific steps of the secondary fusion analysis are as follows:

[0027] Step St1: sort the TTC of multiple obstacle information in Za1 after fusion processing, select the obstacle with the smallest TTC, and proceed to step St3; if there are multiple obstacles with the smallest TTC, proceed to step St2;

[0028] Step St2: Calculate the distance d from each obstacle to the vehicle, and select the obstacle information closest to the vehicle. The calculation formula is shown in Formula 13 below.

[0029]

[0030] Step St3: The obstacle information Za1 obtained in step St1 or St2 and the obstacle information Za2 received by broadcast are judged and processed, and the obstacle with the smallest TTC is selected as Za3; if there are multiple obstacles with the smallest TTC, proceed to step St4;

[0031] Step St4: Calculate the distance between obstacles Za1 and Za2 and the vehicle. The calculation process is the same as step St2. Select the obstacle closest to the vehicle as obstacle information Za3.

[0032] Step St5: Use obstacle information Za3 as input to the vehicle body control module.

[0033] Furthermore, the vehicle body control module can control the vehicle according to the information obtained by the obstacle information processing module, and the control process includes the following steps:

[0034] Step St01: If the vehicle sensor does not detect the obstacle information Za1 and the vehicle communication module does not receive the obstacle information Za2, that is, the obstacle information Za3 does not exist, the vehicle does not take any action;

[0035] Step St02: If the vehicle sensor detects obstacle information but the vehicle communication module does not receive the obstacle information, the obstacle information is first broadcasted, and then the collision time TTC is calculated; based on the obstacle collision time TTC: if TTC is less than the warning time threshold, the driver is reminded on the vehicle dashboard; if TTC is less than the deceleration threshold, the vehicle will pre-decelerate to ensure that the obstacle can pass smoothly; if TTC is less than the emergency braking threshold, the vehicle will make an emergency stop in advance;

[0036] Step St03: If the vehicle does not detect obstacle information but receives obstacle information provided by broadcast, the vehicle will calculate the collision time TTC to determine the action to be performed in step St02;

[0037] Step St04: If the vehicle sensor detects an obstacle and the vehicle communication module receives the obstacle information, the target information is first sorted according to TTC, and then the vehicle is controlled to perform the action required to be executed in step St02 according to the determined obstacle information Za3.

[0038] Furthermore, the collision time TTC is calculated as follows:

[0039] The calculation method of TTC is as follows: T Time to maximum collision TTC max Within, according to the step size t step The position information of the vehicle and each obstacle is predicted according to the laws of kinematics to obtain the position at a certain time in the future t k =t T +k·t step The vehicle position [x(t k ), y(t k )] and the position of the obstacle [ObX(t k ), ObY(t k )]; Assuming that the ego vehicle and each obstacle are rectangular bounding boxes of a specific size, the rectangular bounding boxes of the ego vehicle and the obstacles are overlapped in the x-axis and y-axis directions; if at t k When the rectangular frame of the vehicle overlaps with the rectangular frame of an obstacle, it is considered that there is a collision risk, and the time t at this time is recorded. k It is the collision time TTC of the current obstacle.

[0040] The present invention also discloses a warning method for a V2X-based warning system for pedestrians and two-wheeled vehicles crossing the road, which is characterized by comprising the following steps:

[0041] Step ST1: The environment perception module collects obstacle information Za around the vehicle, judges and identifies the obstacle information Za around the location of the vehicle, and then sends the obstacle information Za1 to the vehicle communication module and the information processing module;

[0042] Step ST2: the vehicle communication module receives the obstacle information Za1 sent by the environment perception module of the vehicle, broadcasts the real-time obstacle information Zal detected by the vehicle to other nearby vehicles, receives the real-time obstacle information Za2 sent by the vehicle communication modules of other vehicles near the vehicle, and sends the real-time obstacle information Za2 to the information processing module;

[0043] Step ST3: the information processing module receives the real-time obstacle information Za1 of the vehicle sent by the environment perception module of the vehicle and the real-time obstacle information Za2 sent by the vehicle communication module, performs secondary fusion analysis on the real-time obstacle information Za1 and the real-time obstacle information Za2, obtains fused obstacle information Za3 after the secondary fusion analysis, and sends the fused obstacle information Za3 to the body control module and the vehicle communication module;

[0044] Step ST4: The vehicle body control module obtains the obstacle information Za3 after comparison and analysis by the information processing module, and controls the vehicle according to the fused obstacle information Za3.

[0045] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0046] The present invention discloses a V2X-based warning system for crossing pedestrians and two-wheeled vehicles and a warning method thereof, comprising an environment perception module, an information processing module, a body control module and an on-board communication module; the environment perception module is arranged on the vehicle and is used to collect obstacle information around the vehicle; the on-board communication module receives obstacle information Za sent by the environment perception module of the vehicle, broadcasts real-time obstacle information Za1 detected by the vehicle and real-time obstacle information Za2 of other vehicles; the information processing module obtains real-time obstacle information Za1 and real-time obstacle information Za2 and fuses them to obtain fused obstacle information Za3; the body control module controls the vehicle according to the fused obstacle information Za3.

[0047] The V2X-based warning system for crossing pedestrians and two-wheeled vehicles and the warning method thereof of the present invention use a data fusion strategy to detect obstacle targets in blind spots, thereby increasing the accuracy of detection and greatly reducing the probability of traffic accidents caused by "ghost probing". There is no need to lay other roadside units on the roadside, and the detection task can be completed by using the vehicle's own on-board information collection device, thus saving a large amount of public resources. In addition, the system can also work in a low-power mode, which can effectively utilize vehicles parked on the roadside and improve the detection capability.

[0048] The V2X-based warning system for crossing pedestrians and two-wheeled vehicles and the warning method thereof of the present invention have the advantages of being able to detect obstacle targets in the visual blind spot in the "ghost poking" situation and reduce the probability of traffic accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The figure is a schematic diagram of the framework of a V2X-based warning system for crossing pedestrians and two-wheeled vehicles according to the present invention.

[0050] Figure 2 The present invention is a schematic diagram of data fusion of a V2X-based warning system for crossing pedestrians and two-wheeled vehicles and a warning method thereof.

[0051] Figure 3 The present invention is a workflow diagram of a V2X-based warning system for crossing pedestrians and two-wheeled vehicles and a warning method thereof.

[0052] Figure 4 It is a schematic diagram of the visual blind area in the "ghost poking" situation in the present invention.

[0053] Figure 5 It is a schematic diagram of the time axis for receiving sensor data in the present invention.

[0054] The present invention will be further described below through specific implementation modes in conjunction with the accompanying drawings. DETAILED DESCRIPTION

[0055] See also Figure 1 to Figure 5 , a V2X-based warning system for pedestrians and two-wheeled vehicles crossing the road of the present invention includes an environment perception module, a vehicle communication module, an information processing module and a body control module;

[0056] The environment perception module is arranged on the vehicle, and the multiple sensors of the environment perception module are used to collect multiple obstacle information Za around the vehicle; the obstacle information Za1 is obtained after a fusion process is performed on the multiple obstacle information Za around the location of the vehicle, and then the obstacle information Za1 is sent to the vehicle communication module and the information processing module;

[0057] The understanding of multiple obstacle information Za is as follows: (1) The multiple information Z1as1, Z2as1, Z3as1, …, Zmas1 of multiple obstacles Z1, Z2, Z3, …, Zm perceived by a single sensor S1 constitutes an obstacle sequence vector [Z1as1, Z2as1, Z3as1, …, Zmas1]; (2) Due to the different sensor installation positions and detection characteristics, the obstacles and the number of obstacles perceived by a single sensor may not be exactly the same, that is, the obstacle sequence vector may contain 0 elements; (3) n sensors S1, S2, S3, …, Sn perceive m obstacles Z1, Z2, Z3, …, Zm, and the obtained obstacle information constitutes an n×m information matrix containing the obstacle sequence vector The information matrix The first row represents the obstacle information obtained by the sensor S1 by detecting obstacles Z1, Z2, Z3, ..., Zm respectively, and the following rows are similar.

[0058] The fusion process is the process of fusing the above information matrix. There may be overlapping detection information of the same obstacle in the n obstacle sequences. The result obtained is the perception information of all obstacles around the vehicle monitored by all sensors of the vehicle.

[0059] The vehicle-mounted communication module receives the obstacle information Za1 sent by the vehicle's environmental perception module, broadcasts the obstacle information Za1 detected by the vehicle to other nearby vehicles through V2X, receives the obstacle information Za2 sent in real time by the vehicle-mounted communication modules of other vehicles with the early warning system near the vehicle, and sends the obstacle information Za2 to the information processing module;

[0060] The information processing module is used to receive the obstacle information Za1 of the vehicle sent by the environment perception module of the vehicle and the obstacle information Za2 received by the vehicle communication module, and perform secondary fusion analysis on the obstacle information Za1 and the obstacle information Za2, and obtain the obstacle information Za3 after the secondary fusion analysis, and then send the obstacle information Za3 to the body control module;

[0061] The vehicle body control module uses the obstacle information Za3 determined by the information processing module and controls the vehicle according to the determined obstacle information Za3.

[0062] In a specific implementation, the environment perception module includes a plurality of sensors for collecting obstacle information Za around the vehicle body, and each sensor is used for collecting a type of obstacle information Za.

[0063] The obstacles include pedestrians crossing the road, two-wheeled vehicles, and other obstacles of different types sensed by the sensor during the current vehicle driving process.

[0064] The obstacle information Za3 is converted into the obstacle information Za3 in the body coordinate system of the vehicle after fusion analysis based on the current position of the vehicle determined by the satellite navigation system, the obstacle information Za1 perceived by the vehicle, and the obstacle information Za2 received through V2X, and then transmitted to the body control module.

[0065] In specific implementation, the sensors include but are not limited to cameras, millimeter wave radars, ultrasonic radars and laser radars. One or more of the sensors are installed at different positions on the vehicle body to accurately sense obstacle information Za within the sensor monitoring range. In specific implementation, one or more of the above sensors are required to participate in the information collection of obstacles around the vehicle, and can accurately sense obstacle information within a certain range around the current vehicle.

[0066] The obstacle information Za1 adopts a target fusion method, which is to fuse the obstacle information Za sensed by multiple different sensors of the vehicle, and then obtain the fused obstacle information Za1 as the obstacle information around the vehicle.

[0067] Furthermore, in order to ensure the consistency of obstacle information due to the different sampling frame rates of different sensors, data alignment needs to be performed using signal synchronization. Signal synchronization is divided into two steps, namely time synchronization and space synchronization. The time synchronization requires the central processing unit to provide a reference time and synchronize the timestamp of each sensor.

[0068] The spatial synchronization requires converting the obstacle information measurements of different sensors in their respective sensor coordinate systems into measurements in the vehicle coordinate system. Establish a vehicle coordinate system, which is defined as: the Z axis is perpendicular to the ground and points upward, the X axis points directly in front of the vehicle's travel direction, the Y axis points from the right side of the vehicle to the left side of the vehicle, and the origin O is located at the projection of the center of the rear wheel axle of the vehicle on the ground. The vehicle coordinate system will change as the vehicle moves, and the obstacle information within the alignment period T is relative to the vehicle coordinate system at the time of reception. Therefore, it is necessary to align the vehicle coordinate system at each alignment time t T , transform the obstacle information in different sensor coordinate systems to the alignment time t T In the ego-vehicle coordinate system, it is convenient to process obstacle information. At each alignment moment t T The self-vehicle coordinate system is used as the coordinate system of the current alignment period T, that is, for the alignment period T, an alignment time t of the alignment period T is taken as the coordinate system of the current alignment period T. T The ego-vehicle coordinate system is used as the ego-vehicle coordinate system for the entire alignment period T. The timeline diagram of receiving sensor data is as follows Figure 5 shown.

[0069] like Figure 5 , assuming that the data alignment period is T, at a certain alignment time tT A set of camera data cam-obj is received at time t1 before, and at a certain alignment time t T A set of radar data radar-obj is received at time t2 before. Both the camera data cam-obj and the radar data radar-obj contain the obstacle position coordinates P at the receiving time. obj and obstacle speed v obj Information, obtain the vehicle yaw rate yawrate and vehicle speed v at the corresponding moment, at the alignment time t T Perform data alignment. The specific steps for data alignment are:

[0070] Step 1: Calculate t i The vehicle yaw angle yaw and posture matrix Rt at the moment are calculated as shown in the following formula (1).

[0071]

[0072]

[0073] where t i Indicates the time when the data is received, which corresponds to t1 or t2 here; in the current cycle, only t1 and t2 are mentioned in the example, and the time when the data is received in the subsequent time axis is t3, t4, t5, etc. (i is incremented by 1); x(t i ) and y(t i ) is t i The coordinates of the vehicle position at the initial time t0 (the initial time t0 represents the time when the vehicle starts) in the vehicle coordinate system are given by the following formula (2).

[0074]

[0075] Step 2: Calculate t T The yaw matrix and Rt matrix of the vehicle at the moment. i Represents the time of receiving data before the alignment time, here t i =t2, then we have the following formulas (3) to (5).

[0076]

[0077]

[0078]

[0079] Step 3: If the obstacle's movement speed v obj is not zero, according to the law of kinematics, the obstacle is predicted at t according to the following formula (6) T The time relative to data reception ti The position of the vehicle in the coordinate system at the moment, P obj is the obstacle position coordinate.

[0080]

[0081] Step 4: Calculate the obstacles received at time t1 and time t2 respectively. T The position coordinate P of the vehicle coordinate system at the time relative to the initial time t0 cam and P radar .in, and are the external parameters calibrated by the vehicle-mounted camera and the vehicle-mounted radar relative to the vehicle coordinate system, respectively, and their expressions are shown in the following equation (7).

[0082]

[0083] Step 5: Based on the t calculated in step 2 T The yaw matrix and Rt matrix of the vehicle at time tT are used to calculate the position coordinates pca of the obstacle in the vehicle coordinate system at time tT. m and Prada r , see the following formula (8).

[0084]

[0085] After completing the above calculations, the obstacle information obtained by different sensors can be converted into a unified vehicle coordinate system, which is convenient for the next calculation.

[0086] In specific implementation, the steps of the environment perception module fusing multiple obstacle information Za around the vehicle are as follows:

[0087] Step 01: Obstacle position velocity vector ObVe of different obstacle information Za measured by the environment perception module; the obstacle position velocity vector ObVe includes the position coordinate ObX of the obstacle in the X-axis direction, the position coordinate ObY of the obstacle in the Y-axis direction, the velocity value ObVx of the obstacle in the X-axis direction, and the velocity value ObVy of the obstacle in the Y-axis direction; that is: ObVe = [ObX, ObY, ObVx, ObVy]; wherein the X-axis and Y-axis are the XY axes of the ego-vehicle coordinate system;

[0088] Step 02: The environment perception module includes multiple sensors, and the perception result output by each sensor is the position and velocity vector ObVe of m obstacles; calculate the cosine similarity S between any two position and velocity vectors ObVe from different sensors;

[0089] n sensors S1, S2, S3, ..., Sn sense m obstacles Z1, Z2, Z3, ..., Zm, and the obstacle information obtained constitutes an n×m information matrix containing the obstacle sequence vector

[0090] The position and velocity vector ObVe1 of the obstacle Z1 obtained by sensor S1, ObVe1 = [ObX1, ObY1, ObVx1, ObVy1]; the position and velocity vector ObVe2 of the obstacle Z1 obtained by sensor S2, ObVe2 = [ObX2, ObY2, ObVx2, ObVy2]; the obstacle position and velocity vector ObVe1 is matched with all obstacle position and velocity vectors obtained by other sensors one by one, and the cosine similarity S between the two obstacle position and velocity vectors ObVe is calculated; if the obtained cosine similarity S>0.8, it is considered that the two obstacle position and velocity vectors ObVe express the same obstacle target, and it is considered that the sensor S1 and the sensor S2 that perceive the obstacle target are matched successfully;

[0091] The cosine similarity of ObVe1 and ObVe2 is calculated as follows: Formula (9) to Formula (11).

[0092]

[0093]

[0094]

[0095] In formula (9) to formula (11), |ObVe1|, |ObVe2| are the lengths of the modulus of the current obstacle position velocity vector ObVe1 or ObVe2, and ObVe1·ObVe2 is the dot product calculation of the two vectors, that is, the corresponding elements of the two vectors are multiplied two by two.

[0096] Step 03: When different sensors sense the same obstacle, the i-th sensor is assigned a weight value W i , the confidence of the i-th sensor is Z i ;According to the matching result of the obstacle information in step 02, obtain the sensor sequence that perceives the obstacle and calculate the comprehensive score value, which is the confidence Z of the obstacle. Z Confidence Z Z The calculation formula is as follows: Formula (12);

[0097]

[0098] In formula (12), n is the number of sensors successfully matched in step 02, i represents the i-th sensor; the weight value W of the obstacle perceived by the i-th sensor is iUsually the allocation is based on expert experience; the more accurate the sensor, the greater the weight value W. i The larger the weight value W i The value range is 0 to 1), and it can also be adjusted dynamically according to the actual scene during specific work. The confidence level Z of a single sensor i It is generally set as the confidence value of the sensor output, which is used to indicate the reliability, validity and relevance of the sensor output.

[0099] Step 04: Obstacle confidence Z obtained in step 03 Z , select the fused obstacles based on the confidence threshold, and filter out the obstacles with confidence Z Z After the obstacle is too low, the reference obstacle Ob is output i ;During specific implementation, the confidence threshold can be set according to actual needs;

[0100] Step 05: Filter the obstacle types, remove obstacles that are not pedestrians or two-wheeled vehicles, and use them as obstacle information Za1 output by the environment perception module. The obstacle position velocity vector of the obstacle information Za1 is ObVe = [ObX, ObY, ObVx, ObVy].

[0101] Step 06: According to the obstacle information Za1 obtained in step 05, add the auxiliary information of the obstacle, including the absolute time Zt of the obstacle discovery, the obstacle type Zl, the confidence of the obstacle information Zz and the position and speed information ObVe of the obstacle; express the obstacle information through a vector, that is, Za1 = [Zt, Zl, Zz, ObVe]. The absolute time Zt is the alignment time t T Corresponding satellite navigation system time.

[0102] In specific implementation, the information processing module can perform secondary fusion analysis on the obstacle information Za1 sensed by the vehicle and the obstacle information Za2 received from other vehicles with the system, and use the obstacle information Za3 after the secondary fusion analysis as the input of the body control module.

[0103] In specific implementation, the information processing module will sort the obstacles according to the collision time TTC (Time to Collision) of the perceived and received obstacle information, and select the obstacle with the smallest TTC as the input of the body control module. If there are multiple obstacles with the same TTC, the obstacle closest to the vehicle position is selected as the input of the body control module;

[0104] The specific steps of the secondary fusion analysis are as follows:

[0105] Step St1: sort the TTC of multiple obstacle information in Za1 after fusion processing, select the obstacle with the smallest TTC, and proceed to step St3; if there are multiple obstacles with the smallest TTC, proceed to step St2;

[0106] Step St2: Calculate the distance d from each obstacle to the vehicle, and select the obstacle information closest to the vehicle. The calculation formula is shown in Formula 13 below.

[0107]

[0108] Step St3: The obstacle information Za1 obtained in step St1 or St2 and the obstacle information Za2 received by broadcast are judged and processed, and the obstacle with the smallest TTC is selected as Za3; if there are multiple obstacles with the smallest TTC, proceed to step St4;

[0109] Step St4: Calculate the distance between obstacles Za1 and Za2 and the vehicle. The calculation process is the same as step St2. Select the obstacle closest to the vehicle as obstacle information Za3.

[0110] Step St5: Use obstacle information Za3 as input to the vehicle body control module.

[0111] In a specific implementation, the vehicle body control module can control the vehicle according to the information obtained by the obstacle information processing module, and the control process includes the following steps:

[0112] Step St01: If the vehicle sensor does not detect the obstacle information Za1 and the vehicle communication module does not receive the obstacle information Za2, that is, the obstacle information Za3 does not exist, the vehicle does not take any action;

[0113] Step St02: If the vehicle sensor detects obstacle information but the vehicle communication module does not receive the obstacle information, the obstacle information is first broadcasted, and then the collision time TTC is calculated; based on the obstacle collision time TTC: if TTC is less than the warning time threshold, the driver is reminded on the vehicle dashboard; if TTC is less than the deceleration threshold, the vehicle will pre-decelerate to ensure that the obstacle can pass smoothly; if TTC is less than the emergency braking threshold, the vehicle will make an emergency stop in advance;

[0114] Step St03: If the vehicle does not detect obstacle information but receives obstacle information provided by broadcast, the vehicle will calculate the collision time TTC to determine the action to be performed in step St02;

[0115] Step St04: If the vehicle sensor detects an obstacle and the vehicle communication module receives the obstacle information, the target information is first sorted according to TTC, and then the vehicle is controlled to perform the action required to be executed in step St02 according to the determined obstacle information Za3.

[0116] In specific implementation, the collision time TTC is calculated as follows:

[0117] The calculation method of TTC is as follows: T Time to maximum collision TTC max Within, according to the step size t step The position information of the vehicle and each obstacle is predicted according to the laws of kinematics to obtain the position at a certain time in the future t k =t T +k·t step The vehicle position [x(t k ), y(t k )] and the position of the obstacle [ObX(t k ), ObY(t k )]; Assuming that the ego vehicle and each obstacle are rectangular bounding boxes of a specific size, the rectangular bounding boxes of the ego vehicle and the obstacles are overlapped in the x-axis and y-axis directions; if at t k When the rectangular frame of the vehicle overlaps with the rectangular frame of an obstacle, it is considered that there is a collision risk, and the time t at this time is recorded. k It is the collision time TTC of the current obstacle.

[0118] The vehicle-mounted communication module adopts C-V2X technology when receiving and sending broadcasts. In the present invention, the technology used for broadcasting is C-V2X (Cellular Vehicle to Everything). The vehicle-mounted communication module can convert the received obstacle information Za2 into a message form that can be recognized by the vehicle itself for use by the body control module. The broadcast information will be verified for reliability before use. The broadcast transmission cycle is 20ms and the broadcast duration is 100ms. Reliability verification is an asymmetric encryption verification method used to prevent the reception of messy information and affect the normal driving of the vehicle.

[0119] The asymmetric encryption verification method is a method to ensure information security and reliability. When the system receives obstacle information, it will use the secret key to verify the reliability of the signal at the same time. If the verification fails, the signal is considered unsafe and will not enter the body control module. If the verification is successful, the encrypted information will be decrypted and decrypted into valid obstacle information, which will be passed to the information processing module for processing. The present invention adopts the RSA encryption algorithm, and the encryption method used is asymmetric encryption, that is, the encryption key and the decryption key are different, and the encryption key cannot be calculated by the decryption key. This can fully ensure the security of the source of obstacle information, thereby ensuring the safe driving of the driver.

[0120] The present invention also discloses a warning method for a V2X-based warning system for pedestrians and two-wheeled vehicles crossing the road, which is characterized by comprising the following steps:

[0121] Step ST1: The environment perception module collects obstacle information Za around the vehicle, judges and identifies the obstacle information Za around the location of the vehicle, and then sends the obstacle information Za1 to the vehicle communication module and the information processing module;

[0122] Step ST2: the vehicle communication module receives the obstacle information Za1 sent by the environment perception module of the vehicle, broadcasts the real-time obstacle information Za1 detected by the vehicle to other nearby vehicles, receives the real-time obstacle information Za2 sent by the vehicle communication modules of other vehicles near the vehicle, and sends the real-time obstacle information Za2 to the information processing module;

[0123] Step ST3: the information processing module receives the real-time obstacle information Za1 of the vehicle sent by the environment perception module of the vehicle and the real-time obstacle information Za2 sent by the vehicle-mounted communication module, performs secondary fusion analysis on the real-time obstacle information Za1 and the real-time obstacle information Za2, obtains fused obstacle information Za3 after the secondary fusion analysis, and sends the fused obstacle information Za3 to the body control module and the vehicle-mounted communication module;

[0124] Step ST4: The vehicle body control module obtains the obstacle information Za3 after comparison and analysis by the information processing module, and controls the vehicle according to the fused obstacle information Za3.

[0125] In the V2X-based pedestrian and two-wheeled vehicle crossing warning system and its warning method, during the normal driving of the vehicle, the information sending process and the information receiving process are run simultaneously, and the mutual exchange of data information will not be affected by each other. Figure 3 As shown, the workflow includes the following steps.

[0126] S1: The vehicle starts, each module is initialized, enters the working state, and enters S2.

[0127] S2: The vehicle-mounted information collection device senses obstacles around the vehicle and enters S3.

[0128] S3: The system integrates the information of obstacles sensed by each vehicle-mounted information collection device and enters S4.

[0129] S4: The system judges the fused obstacle information, first judging whether the obstacle is located in front of or to the side of the vehicle's travel direction. If so, it proceeds to S5; otherwise, it proceeds to S7.

[0130] S5: Further determine whether there is an obstacle whose confidence is greater than the confidence threshold and whose type is a pedestrian or a two-wheeled obstacle. If so, proceed to S6; otherwise, proceed to S7.

[0131] S6: The communication module uses V2X technology to broadcast obstacle information to surrounding vehicles at a frequency of 20ms. The broadcast content includes: the time, type, speed, location of the obstacle, and the confidence level of the obstacle after data fusion, etc., and then enters S7.

[0132] S7: After the broadcast ends, the communication subsystem enters the receiving state and enters S8. The information collection device continues to detect obstacles around the vehicle body at a frequency of 20ms and enters S2.

[0133] S8: Check whether there are other vehicles broadcasting obstacle information. If yes, go to S9; otherwise, go to S10.

[0134] S9: The communication subsystem receives the broadcast of other obstacle information and enters S10.

[0135] S10: Perform TTC sorting on the perceived and received obstacle information. If yes, proceed to S11; otherwise, proceed to S12.

[0136] S11: Select the obstacle with the smallest TTC as the input of the body control module. If there are multiple obstacles with the same TTC, select the obstacle closest to the vehicle and enter S12.

[0137] S12: The body control module controls the vehicle according to the obstacle information, based on the collision time TTC of the obstacle: if TTC is less than the warning time threshold, the driver will be reminded on the dashboard of the vehicle; if TTC is less than the deceleration threshold, the vehicle will pre-decelerate to ensure that the obstacle can pass smoothly; if TTC is less than the emergency braking threshold, the vehicle will make an emergency stop in advance and enter S13. If the above conditions are not met, no control action will be performed and enter S8.

[0138] S13: The communication subsystem sends a warning message to the dashboard of the vehicle to prompt the driver, and then enters S14.

[0139] S14: After waiting for the obstacle to pass smoothly, restart the vehicle and continue to detect whether there is any other obstacle information broadcast, and enter S8.

[0140] like Figure 4 As shown, the shadow area in front of car A is its sensing area. When pedestrians or two-wheeled vehicles cross in front of car B, car A is limited by its sensing area and cannot timely obtain information about obstacles such as pedestrians or two-wheeled vehicles that are blocked by car A. At this time, if car B sends the information about obstacles such as pedestrians or two-wheeled vehicles detected by it to other traffic participants, such as car A, through the vehicle-mounted communication module, the system will warn the driver on the dashboard of car A. If the speed of car A is greater than that of car B at this time, the body control module will pre-decelerate car A, which not only reduces the possibility of a collision between the vehicle and pedestrians or two-wheeled vehicles, but also effectively avoids damage to the vehicle caused by emergency braking and poor driving experience for drivers and passengers, thereby reducing the probability of traffic accidents.

[0141] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

[0142] In addition, it should be understood that although the present specification is described according to implementation modes, not every implementation mode contains only one independent technical solution. This description of the specification is only for the sake of clarity. Those skilled in the art should regard the specification as a whole. The technical solutions in each embodiment may also be appropriately combined to form other implementation modes that can be understood by those skilled in the art.

Claims

1. A V2X-based warning system for pedestrians and two-wheeled vehicles crossing the road, characterized by: Includes environmental perception module, vehicle communication module, information processing module and body control module; The environment perception module is provided on the vehicle, and the multiple sensors of the environment perception module are used to collect multiple obstacle information Za around the vehicle; the obstacle information Za1 obtained after a fusion process of the multiple obstacle information Za around the vehicle is obtained, and then the obstacle information Za1 is sent to the vehicle communication module and the information processing module; The vehicle-mounted communication module receives obstacle information Za1 sent by the vehicle's environmental perception module, broadcasts the obstacle information Za1 detected by the vehicle to other nearby vehicles via V2X, and receives obstacle information Za2 sent in real time by the vehicle-mounted communication modules of other nearby vehicles equipped with the warning system, and sends the obstacle information Za2 to the information processing module. The information processing module is configured to receive the obstacle information Za1 of the vehicle sent by the vehicle's environment perception module and the obstacle information Za2 received by the vehicle-mounted communication module, perform a secondary fusion analysis on the obstacle information Za1 and the obstacle information Za2, obtain the obstacle information Za3 after the secondary fusion analysis, and then send the obstacle information Za3 to the body control module; The vehicle body control module uses the obstacle information Za3 determined by the information processing module and controls the vehicle according to the determined obstacle information Za3.

2. A V2X-based pedestrian and two-wheeled vehicle crossing warning system according to claim 1, characterized in that: The environment perception module includes a plurality of sensors for collecting obstacle information Za around the vehicle body, and each sensor is used to collect a type of obstacle information Za.

3. The V2X-based pedestrian and two-wheeled vehicle crossing warning system according to claim 2, characterized in that: The sensors include but are not limited to cameras, millimeter-wave radars, ultrasonic radars, and lidars.

4. The V2X-based pedestrian and two-wheeled vehicle crossing warning system according to claim 1 is characterized in that: The steps of the environment perception module fusing multiple obstacle information Za around the vehicle are as follows: Step 01: Obtain the obstacle position and velocity vector ObVe of the different obstacle information Za measured by the environment perception module. The obstacle position and velocity vector ObVe includes the obstacle's X-axis position coordinate ObX, the obstacle's Y-axis position coordinate ObY, the obstacle's X-axis velocity ObVx, and the obstacle's Y-axis velocity ObVy. Specifically, ObVe = [ObX, ObY, ObVx, ObVy], where the X and Y axes are the X and Y axes of the ego-vehicle coordinate system. Step 02: The environment perception module includes multiple sensors. Each sensor outputs the perception result of m obstacle position and velocity vectors ObVe. Calculate the cosine similarity S between any two position and velocity vectors ObVe from different sensors. Step 03: When different sensors perceive the same obstacle, the i-th sensor is assigned a weight value W i , the confidence of the i-th sensor is Z i According to the matching result of the obstacle information in step 02, the sensor sequence that perceives the obstacle is obtained and the comprehensive score is calculated, which is the confidence Z of the obstacle. Z Confidence Z Z The calculation formula is as follows: Formula (12); In formula (12), n is the number of sensors successfully matched in step 02, and i represents the i-th sensor; Step 04: Obstacle confidence Z obtained in step 03 Z , select the fused obstacles based on the confidence threshold, and filter out the obstacles with confidence Z Z After the obstacle is too low, the reference obstacle Ob is output i ; Step 05: Filter the obstacle types and remove obstacles that are not pedestrians or two-wheeled vehicles. This is used as the obstacle information Za1 output by the environment perception module. Step 06: Based on the obstacle information Za1 obtained in step 05, additional information about the obstacle is added, including the absolute time Zt when the obstacle was discovered, the obstacle type Zl, the confidence level Zz of the obstacle information, and the position and speed information ObVe of the obstacle.

5. The V2X-based pedestrian and two-wheeled vehicle crossing warning system according to claim 1 is characterized in that: The information processing module can perform secondary fusion analysis on the obstacle information Za1 sensed by the vehicle and the obstacle information Za2 received from other vehicles equipped with the system, and use the obstacle information Za3 after the secondary fusion analysis as the input of the body control module.

6. The V2X-based pedestrian and two-wheeled vehicle crossing warning system according to claim 1, characterized in that: The information processing module ranks obstacles based on their perceived and received time to collision (TTC) and selects the obstacle with the smallest TTC as input to the body control module. If multiple obstacles have the same TTC, the obstacle closest to the vehicle is selected as input to the body control module. The specific steps of the secondary fusion analysis are as follows: Step St1: Sort the TTC of the multiple obstacle information in Za1 after fusion processing, select the obstacle with the minimum TTC, and proceed to step St3; if there are multiple obstacles with the minimum TTC, proceed to step St2; Step St2: Calculate the distance d from each obstacle to the vehicle, and select the obstacle closest to the vehicle. The calculation formula is shown in Formula 13 below. Step St3: The obstacle information Za1 obtained in step St1 or St2 and the obstacle information Za2 received by broadcast are judged and processed, and the obstacle with the smallest TTC is selected as Za3; If there are multiple obstacles with the smallest TTC, go to step St4; Step St4: Calculate the distance between obstacles Za1 and Za2 and the vehicle. The calculation process is the same as step St2. The obstacle closest to the vehicle is selected as obstacle information Za3. Step St5: Obstacle information Za3 is used as input to the vehicle body control module.

7. The V2X-based pedestrian and two-wheeled vehicle crossing warning system according to claim 1 is characterized in that: The vehicle body control module can control the vehicle based on the information obtained by the obstacle information processing module. The control process includes the following steps: Step St01: If the vehicle sensor does not detect the obstacle information Za1 and the vehicle communication module does not receive the obstacle information Za2, that is, the obstacle information Za3 does not exist, the vehicle does not take any action; Step St02: If the vehicle sensor detects obstacle information but the vehicle communication module does not receive the obstacle information, the obstacle information is first broadcast and then the collision time TTC is calculated; Based on the obstacle's time to collision (TTC): If the TTC is less than the warning time threshold, the driver will be reminded on the vehicle's dashboard; If the TTC is less than the deceleration threshold, the vehicle will pre-decelerate to ensure that the obstacle can be passed smoothly; if the TTC is less than the emergency braking threshold, the vehicle will make an emergency stop in advance; Step St03: If the vehicle does not detect obstacle information but receives obstacle information provided by broadcast, the vehicle will calculate the time to collision (TTC) to determine the action to be performed in step St02; Step St04: If the vehicle sensor detects an obstacle and the vehicle communication module receives the obstacle information, the target information is first sorted according to the TTC, and then the vehicle is controlled to perform the action required to be executed in step St02 based on the determined obstacle information Za3.

8. A V2X-based pedestrian and two-wheeled vehicle crossing warning system according to claim 6 or 7, characterized in that: The collision time TTC is calculated as follows: TTC is calculated by predicting the position of the ego vehicle and each obstacle based on the laws of kinematics, with a step size of tstep, from the current time tT to the maximum collision time TTCmax. This results in the ego vehicle's position [x(tk), y(tk)] and the obstacle's position [ObX(tk), ObY(tk)] at a future time tk = tT + k·tstep. Assuming that the ego vehicle and each obstacle are rectangular bounding boxes of a specific size, the ego vehicle's rectangular bounding box and the obstacle's rectangular bounding box are checked for overlap along the x- and y-axes. If the ego vehicle's rectangular bounding box overlaps with an obstacle's rectangular bounding box at time tk, a collision risk is considered and the time tk at that time is recorded as the current obstacle's collision time TTC.

9. A warning method for a V2X-based pedestrian and two-wheeled vehicle crossing warning system according to claim 1, characterized in that: The steps include: Step ST1: The environment perception module collects obstacle information Za around the vehicle, determines and identifies the obstacle information Za around the vehicle, and then sends the obstacle information Za1 to the vehicle communication module and the information processing module; Step ST2: The vehicle-mounted communication module receives the obstacle information Za1 sent by the vehicle's environmental perception module, broadcasts the real-time obstacle information Za1 detected by the vehicle to other nearby vehicles, receives the real-time obstacle information Za2 sent by the vehicle-mounted communication modules of other nearby vehicles, and sends the real-time obstacle information Za2 to the information processing module. Step ST3: The information processing module receives the real-time obstacle information Za1 of the vehicle sent by the vehicle's environment perception module and the real-time obstacle information Za2 sent by the vehicle-mounted communication module, performs a secondary fusion analysis on the real-time obstacle information Za1 and the real-time obstacle information Za2, obtains fused obstacle information Za3 after the secondary fusion analysis, and sends the fused obstacle information Za3 to the body control module and the vehicle-mounted communication module; Step ST4: The vehicle body control module obtains the obstacle information Za3 obtained by the information processing module after comparison and analysis, and controls the vehicle according to the fused obstacle information Za3.

Images