Commercial vehicle collision danger early warning method based on multi-mode fusion perception technology

Abstract

The invention discloses a commercial vehicle collision danger early warning method based on a multi-mode fusion perception technology. The method specifically comprises the following steps: S1, path planning; s2, road junction preloading is carried out; s3, vehicle accompanying danger judgment; s4, danger early warning is carried out; the invention relates to the technical field of automobile safety driving. According to the commercial vehicle collision danger early warning method based on the multi-mode fusion perception technology, road condition information of an intersection, dynamic moving object information of the intersection and real-time displacement information of the commercial vehicle and a moving object in a driving route of the commercial vehicle are subjected to fusion measurement and calculation, the collision risk is accurately pre-judged, and collision early warning is sent to a driver in advance; according to the method, the driver is assisted to take measures to solve the collision risk, the method has the advantages of preposition and accuracy of early warning, meanwhile, false risk information can be effectively filtered out, the early warning frequency is greatly reduced, the driver can be helped to eliminate stress passivation reaction to frequent early warning, and the real-time responsivity to early warning is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of automobile safe driving, and in particular to a commercial vehicle collision risk warning method based on multi-modal fusion sensing technology. Background Art

[0002] Traditional vehicle blind spot detection algorithms have too many alarms, and drivers are insensitive to alarm stress, so blind spot alarms do not really serve as a reminder. In addition, with the widespread use of electric vehicles, such as electric bicycles and electric motorcycles, electric vehicle accidents in the rear blind spot of vehicles have greatly increased.

[0003] Since there is usually only a 2-second reaction time from when a vehicle starts turning to when it collides with an electric vehicle, the traditional single-camera blind spot detection algorithm does not have enough time to alert the driver, or even if it alerts the driver, it is too late to react, and cannot effectively solve the problem of a large truck turning right and colliding with an electric vehicle going straight. Large trucks and other commercial vehicles also have similar collision problems when changing lanes or turning left, and traditional blind spot detection algorithms cannot effectively solve them.

[0004] Based on the retrieval of the above information, a commercial vehicle collision risk warning method based on multimodal fusion perception technology is proposed. At the intersection where commercial vehicles turn right, potential dangerous events of electric vehicles on the right side are tracked and judged in advance, and effective reminders are given to drivers, which can effectively reduce the probability of collision accidents between electric vehicles, cars and pedestrians and commercial vehicles. Summary of the invention

[0005] 1. Technical issues to be resolved

[0006] In view of the shortcomings of the prior art, the present invention provides a commercial vehicle collision hazard warning method based on multimodal fusion sensing technology, which solves the problem that the existing commercial vehicle blind spot monitoring alarms are frequent, the driver cannot be effectively screened and warned, and the driver is easily fatigued by the equipment alarm response and stress blunted, and an effective warning effect cannot be achieved.

[0007] (II) Technical solution

[0008] To achieve the above objectives, the present invention is implemented through the following technical solutions: a commercial vehicle collision risk warning method based on multimodal fusion sensing technology, specifically comprising the following steps:

[0009] S1. Path planning: Obtain the path planning of commercial vehicles and obtain the driving trajectory in advance;

[0010] S2, intersection preloading: According to the driving trajectory, the location information of the intersection to be turned is extracted to calculate the time required for the commercial vehicle to arrive at the intersection. When the preset time threshold is reached, the moving objects on both sides of the commercial vehicle are tracked. The tracking method includes but is not limited to the machine vision algorithm of the camera and the reflection perception algorithm of the actively emitted radar wave;

[0011] S3. Determination of accompanying vehicle risk: Based on the target tracking results, the accompanying time of the moving object and the commercial vehicle, the risk factor of the distance change between the moving object and the vehicle are calculated, and the multi-modal and multi-dimensional risk factors are integrated and calculated to classify and determine the collision risk of the moving object;

[0012] S4. Dangerous advance warning: When a commercial vehicle is driving, the remaining time for the commercial vehicle to reach the turning intersection and possibly collide with a moving object when turning is calculated in real time. When the remaining time reaches the alarm threshold, the driver is given a corresponding warning based on the danger classification result of the corresponding moving object.

[0013] The present invention is further configured such that when calculating the time required for the commercial vehicle to arrive at the right-turn intersection in S2, the calculation is performed based on the current speed of the commercial vehicle.

[0014] The present invention is further configured as follows: the moving objects in S2 include electric bicycles, electric motorcycles, cars and pedestrians.

[0015] The present invention is further configured as follows: the method of determining the danger classification of the moving object in S3 includes:

[0016] The danger level of the moving object is comprehensively determined based on multi-modal and multi-dimensional factors such as the length of time the moving object and the commercial vehicle are accompanied, and the change in the distance between the moving object and the vehicle.

[0017] The present invention is further configured as follows: the risk levels include Grade A, Grade B and Grade C in descending order.

[0018] in:

[0019] Class A is long-term accompanying time that reaches the preset long accompanying time value at the intersection;

[0020] Class B is short-time accompanying that reaches the preset short accompanying time value at the intersection;

[0021] Level C means that before reaching the intersection, there are no moving objects outside the vehicle and there is no danger.

[0022] The present invention is further configured as follows: the method for calculating in real time the remaining time that a commercial vehicle may collide with a moving object when turning or merging at an intersection in S4 includes:

[0023] Get the current moving speed of the moving object and the current moving speed of the commercial vehicle, calculate the collision time of a possible collision based on the turning distance from the commercial vehicle, taking the right turn distance as an example, and calculate the remaining time based on the current position of the commercial vehicle.

[0024] The present invention is further configured as follows: in said S4, according to the result of the danger classification determination of the corresponding moving object, the manner of issuing a corresponding warning to the driver includes:

[0025] When the danger level of the moving object is Class A, the highest level danger warning is issued to the driver;

[0026] When the danger level of the moving object is Class B, a normal voice prompt warning is issued to the driver;

[0027] When the danger level of the moving object is Class C, no warning is issued to the driver.

[0028] The present invention is further configured such that: the danger warning includes sound and light reminders and voice prompts inside or outside the vehicle.

[0029] (III) Beneficial effects

[0030] The present invention provides a commercial vehicle collision risk warning method based on multi-modal fusion sensing technology. It has the following beneficial effects:

[0031] The present invention obtains information about the intersection to be turned in advance according to the driving route of the commercial vehicle, and uses the time required for the commercial vehicle to arrive at the intersection as the starting time for tracking the moving object. The danger level of the moving object is comprehensively determined by multi-modal and multi-dimensional factors through the length of time the moving object and the commercial vehicle accompany each other and the change in the distance between the moving object and the vehicle. After determining the remaining time for a possible collision between the commercial vehicle and the moving object, the driver is accurately warned according to the danger level classification result, and different forms of warnings with different danger levels are issued while reducing the warning frequency, effectively eliminating the driver's stress passivation and providing reliable protection for reducing the probability of collision accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 A simulation schematic diagram of a commercial vehicle and a moving object in an embodiment of the present invention;

[0033] Figure 2 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0035] See also Figure 1-2 , the embodiment of the present invention provides the following technical solutions:

[0036] Embodiment 1

[0037] A commercial vehicle collision risk warning method based on multimodal fusion sensing technology specifically includes the following steps:

[0038] S1. Path planning: Obtain the path planning of commercial vehicles and obtain the driving trajectory in advance;

[0039] S2, intersection preloading: according to the driving trajectory, the location information of the intersection to be turned is extracted to calculate the time required for the commercial vehicle to arrive at the intersection. When the preset time threshold is reached, the moving objects on both sides of the commercial vehicle are tracked. The tracking method includes but is not limited to the machine vision algorithm of the camera and the reflection perception algorithm of the actively emitted radar wave. The time required for the commercial vehicle to arrive at the intersection to be turned is calculated according to the current speed of the commercial vehicle. Taking the time required for a right turn intersection as an example, when the preset time threshold DT1 is reached, it is recorded as time T0, and the moving objects on the right side of the commercial vehicle, including electric bicycles, electric motorcycles, cars and pedestrians, are tracked;

[0040] As a detailed description, the preset time threshold DT1 is set to 30s by default and can be pre-adjusted according to actual needs;

[0041] S3. Determination of accompanying vehicle danger: Based on the target tracking results, the risk factors including the accompanying time of the moving object and the commercial vehicle and the change in the distance between the moving object and the vehicle can be calculated. These multi-modal and multi-dimensional risk factors are integrated and calculated to classify and determine the collision risk of the moving object. According to the accompanying time of the moving object and the commercial vehicle, after the long accompanying time reaches the preset long accompanying time, the danger level of the moving object is determined to be Class A;

[0042] S4. Danger warning in advance: When the commercial vehicle is driving, the current moving speed of the moving object and the current moving speed of the commercial vehicle are obtained, and the possible collision time T2 is calculated according to the right turning distance from the commercial vehicle, and the remaining time DT2 is calculated according to the current position of the commercial vehicle. Figure 1 As shown, DT2 = T2-T1, where T1 is the time to complete the dangerousness determination of the moving object. When the remaining time DT2 reaches the alarm threshold, the highest level of dangerous warning is issued to the driver, such as flashing of the equipment inside the car, long alarm sounding, and sound and light reminders and voice prompts are issued outside the car at the same time;

[0043] As a detailed explanation, the alarm threshold is set to 10s by default and can be pre-adjusted according to actual needs.

[0044] Embodiment 2

[0045] A commercial vehicle collision risk warning method based on multimodal fusion sensing technology specifically includes the following steps:

[0046] S1. Path planning: Obtain the path planning of commercial vehicles and obtain the driving trajectory in advance;

[0047] S2, intersection preloading: according to the driving trajectory, the location information of the intersection to be turned is extracted to calculate the time required for the commercial vehicle to arrive at the intersection. When the preset time threshold is reached, the moving objects on both sides of the commercial vehicle are tracked. The tracking method includes but is not limited to the machine vision algorithm of the camera and the reflection perception algorithm of the actively emitted radar wave. The time required for the commercial vehicle to arrive at the intersection to be turned is calculated according to the current speed of the commercial vehicle. Taking the time required for a right turn intersection as an example, when the preset time threshold DT1 is reached, it is recorded as time T0, and the moving objects on the right side of the commercial vehicle, including electric bicycles, electric motorcycles, cars and pedestrians, are tracked;

[0048] As a detailed description, the preset time threshold DT1 is set to 30s by default and can be pre-adjusted according to actual needs;

[0049] S3. Determination of accompanying vehicle danger: Based on the target tracking results, the risk factors including the accompanying time length of the moving object and the commercial vehicle and the distance change between the moving object and the vehicle can be calculated. These multi-modal and multi-dimensional risk factors are integrated and calculated to classify and determine the collision risk of the moving object. According to the accompanying time length of the moving object and the commercial vehicle, after the short accompanying time reaches the preset short accompanying time, the danger level of the moving object is determined to be Class B;

[0050] S4. Danger warning in advance: When the commercial vehicle is driving, the current moving speed of the moving object and the current moving speed of the commercial vehicle are obtained, and the possible collision time T2 is calculated according to the right turning distance from the commercial vehicle, and the remaining time DT2 is calculated according to the current position of the commercial vehicle. Figure 1 As shown, DT2 = T2-T1, where T1 is the time to complete the dangerousness determination of the moving object. When the remaining time DT2 reaches the alarm threshold, a normal voice prompt warning is issued to the driver, such as "a collision with an electric vehicle may occur at the intersection ahead";

[0051] As a detailed explanation, the alarm threshold is set to 10s by default and can be pre-adjusted according to actual needs.

[0052] Embodiment 3

[0053] A commercial vehicle collision risk warning method based on multimodal fusion sensing technology specifically includes the following steps:

[0054] S1. Route planning: According to the waybill information of the commercial vehicle, the driving route of the commercial vehicle is obtained and the location information of the intersection to be turned is extracted;

[0055] S2, intersection preloading: Calculate the time required for the commercial vehicle to reach the intersection to be turned based on the current speed of the commercial vehicle. Take the right turn as an example. When the preset time threshold DT1 is reached, it is recorded as time T0, and the moving objects including electric bicycles, electric motorcycles, cars and pedestrians on the right side of the commercial vehicle are tracked;

[0056] As a detailed description, the preset time threshold DT1 is set to 30s by default and can be pre-adjusted according to actual needs;

[0057] S3. Assessing the danger of accompanying vehicles: When the moving object no longer accompanies the commercial vehicle, the danger level of the moving object is determined to be Class C. At this time, there is no danger and no warning is issued to the driver.

[0058] In summary, the present invention can effectively reduce the number of blind spot alarms, can truly alarm for truly dangerous situations, greatly improve the effect of alarms, and avoid major personal injury accidents.

[0059] As an expanded explanation, even if there is no destination information and the complete driving route is not known, the map information can be introduced into the present invention, combined with the position information of the commercial vehicle and the external target information perceived by the camera, to also remind people of possible right turn collisions at the intersection ahead, thereby achieving a certain early warning effect.

[0060] In summary, the present invention overcomes the narrowness and lag of single information by integrating multiple information, conducting systematic sampling and analysis of risk factors, and accurately calculating through big data models, thereby enhancing the pre-emptive and accurate nature of early warnings, and helping drivers to take measures in advance to effectively avoid collision risks. At the same time, it can also effectively filter out false risk information and significantly reduce the frequency of early warnings, helping drivers to eliminate stress-induced blunting reactions to frequent warnings, and improving real-time responsiveness to early warnings, thereby realizing early detection of accident hazards and early alarms, and providing a reliable means for commercial vehicles to reduce road traffic accidents.

[0061] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0062] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A commercial vehicle collision risk warning method based on multi-modal fusion perception technology, characterized by: The specific steps include: S1. Path planning: Obtain the path planning of commercial vehicles and obtain the driving trajectory in advance; S2, intersection preloading: According to the driving trajectory, the location information of the intersection to be turned is extracted to calculate the time required for the commercial vehicle to arrive at the intersection. When the preset time threshold is reached, the moving objects on both sides of the commercial vehicle are tracked; S3. Determination of accompanying vehicle risk: Based on the target tracking results, the accompanying time of the moving object and the commercial vehicle, the risk factor of the distance change between the moving object and the vehicle are calculated, and the multi-modal and multi-dimensional risk factors are integrated and calculated to classify and determine the collision risk of the moving object; S4. Dangerous advance warning: When a commercial vehicle is driving, the remaining time for the commercial vehicle to reach the turning intersection and collide with a moving object when turning is calculated in real time. When the remaining time reaches the alarm threshold, the driver is given a corresponding warning based on the danger classification result of the corresponding moving object.

2. The commercial vehicle collision risk warning method based on multi-modal fusion sensing technology according to claim 1 is characterized by: When calculating the time required for the commercial vehicle to arrive at the intersection in S2, the calculation is performed based on the current speed of the commercial vehicle.

3. The commercial vehicle collision risk warning method based on multi-modal fusion sensing technology according to claim 1 is characterized by: The moving objects in S2 include electric bicycles, electric motorcycles, cars and pedestrians.

4. The commercial vehicle collision risk warning method based on multi-modal fusion sensing technology according to claim 1 is characterized by: The method of determining the danger classification of the moving object in S3 includes: Based on the length of time that the moving object and the commercial vehicle accompany each other and the change in the distance between the target and the vehicle as risk factors, the multi-modal and multi-dimensional risk factors are fused and calculated to comprehensively determine the collision risk of the moving object.

5. The commercial vehicle collision risk warning method based on multi-modal fusion sensing technology according to claim 4 is characterized by: The risk levels include A, B and C from high to low, among which: Class A is long-term accompanying time that reaches the preset long accompanying time value at the intersection; Class B is short-time accompanying that reaches the preset short accompanying time value at the intersection; Level C means that before reaching the intersection, there are no moving objects outside the vehicle and there is no danger.

6. The commercial vehicle collision risk warning method based on multi-modal fusion sensing technology according to claim 5 is characterized by: The method of calculating the remaining time that the commercial vehicle may collide with the moving object in real time when turning in S4 includes: The current moving speed of the moving object and the current moving speed of the commercial vehicle are obtained, and the collision time when a collision may occur is calculated based on the turning distance from the commercial vehicle to the intersection. The remaining time is continuously calculated based on the current position change of the commercial vehicle.

7. The commercial vehicle collision risk warning method based on multi-modal fusion sensing technology according to claim 6 is characterized by: In S4, the method of issuing a corresponding warning to the driver according to the result of the danger classification of the corresponding moving object includes: When the danger level of the moving object is Class A, the highest level danger warning is issued to the driver; When the danger level of the moving object is Class B, a voice prompt warning will be issued to the driver; When the danger level of the moving object is Class C, no warning is issued to the driver.

8. The commercial vehicle collision risk warning method based on multi-modal fusion sensing technology according to claim 7 is characterized by: The hazard warning includes sound and light reminders and voice prompts inside and outside the vehicle.

Images