Lane departure early warning method and device based on high-precision map and multi-source sensor fusion data
Through the lane departure warning system with high-precision map and multi-source sensor fusion data, the problems of low recognition accuracy and high hardware cost in traditional systems are solved, intelligent identification and active intervention in different road sections and vehicle speeds are achieved, and lane departure warning is improved. accuracy and reliability.
Patent Information
- Application Number
- CN202411092869.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-08
- Publication Date
- 2024-12-06
AI Technical Summary
Traditional lane departure warning systems are affected by the environment and equipment sensitivity, and the identification accuracy is difficult to ensure, the hardware cost is high, and it is difficult to identify and determine dangerous areas and sudden high-risk driving behaviors under different driving routes and speeds.
The lane departure warning method based on high-precision maps and multi-source sensor fusion data is adopted. Multi-channel sensor data is processed through the processor, combined with the vehicle CAN bus to achieve active intervention and control, and the lane is collected using angle radar, lidar, millimeter wave radar, and cameras. Environmental information, positioning module is used for satellite positioning, storage module pre-stores map data, alarm module issues sound and light warning, communication module connects external devices, steering wheel steering sensor and acceleration sensor are used for vehicle data collection, realizing the data of different road sections and vehicle speeds Intelligent identification and active intervention.
It can distinguish dangerous areas of the travel route and warning in advance, intelligently identify and determine lane deviation behaviors at different sections and speeds, and actively identify and intervene in sudden high-risk driving behaviors, which improves the accuracy and reliability of lane deviation warning.
Smart Images

Figure CN119091683A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of assisted driving, and in particular relates to a lane departure warning method and device based on high-precision maps and multi-source sensor fusion data. Background Art
[0002] The lane departure warning system can collect data in real time through the intelligent perception of multiple sensor devices, and issue danger warnings in a timely manner, thereby reducing traffic accidents caused by lane departure. Traditional lane departure warning systems are mostly based on identification and judgment based on on-board detection equipment such as cameras, millimeter-wave radars, and lidars. They are easily affected by factors such as the environment and equipment sensitivity. It is difficult to ensure identification accuracy, the hardware cost is high, and it is difficult to identify and judge dangerous sections of the driving route. It is also difficult to identify and actively intervene in the sudden turning behavior of vehicles on different driving sections and at different speeds.
[0003] In response to the above problems, the present invention proposes a lane departure warning method and device based on high-precision maps and multi-source sensor fusion data, which are used to distinguish dangerous sections of the route and provide early warnings, intelligently identify and determine lane departure behaviors in different sections and at different vehicle speeds, and actively identify and intervene in sudden high-risk driving behaviors. Summary of the invention
[0004] The purpose of this patent is to provide a lane departure warning method and device based on high-precision maps and multi-source sensor fusion data. The method and device are used to distinguish dangerous sections of the route and provide early warning, intelligently identify and determine lane departure behaviors in different sections and at different vehicle speeds, and actively identify and intervene in sudden high-risk driving behaviors.
[0005] The technical solution adopted by the present invention to solve the technical problem is achieved as follows:
[0006] The processor is used for processing and fusing data collected by multiple sensors, and is connected to the vehicle CAN bus to achieve active intervention and control of the vehicle.
[0007] The corner radar, laser radar, millimeter wave radar, and camera are used to collect lane environment information, and the processor performs data fusion to determine and identify the lane line data information, traffic sign information, and environmental data information in front of the vehicle.
[0008] The positioning module is used for collecting and processing vehicle satellite positioning data.
[0009] The storage module is used for storing and caching data information and can pre-store vehicle-mounted map data information.
[0010] The alarm module receives the processor data instruction and issues sound and light alarm information.
[0011] The communication module is used for accessing external devices and receiving information. The system does not rely on vehicle-mounted hardware devices and can establish connections with mobile phones and external terminal devices through the communication module to achieve access to high-precision maps and positioning data.
[0012] The steering wheel steering sensor and acceleration sensor / gyroscope are used to collect data on the vehicle's own speed, steering angle, and steering wheel inclination.
[0013] Furthermore, a lane departure warning method based on high-precision maps and multi-source sensor fusion data includes the following steps:
[0014] Step S1: Turn on the vehicle satellite positioning and receive positioning information; the vehicle positioning information can be based on the vehicle's own satellite positioning device, or the vehicle positioning can be achieved by connecting an external device through the communication module;
[0015] Step S2: Based on the vehicle satellite positioning information, determine whether the positioning accuracy is high-precision positioning and whether high-precision map assistance has been enabled. If yes, proceed to step S3; otherwise, proceed to step S2.1;
[0016] Step S2.1: In the absence of high-precision maps and high-precision positioning, further determine whether there is in-vehicle map assistance, and the in-vehicle map data can be connected to an external device through the communication module. If yes, proceed to step S2.1.1, otherwise, proceed to step S2.1.2;
[0017] Step S2.1.1: With the assistance of the on-board map, the multi-source sensor data of the vehicle is integrated to assist in determining the lane line position information, achieve lane-level high-precision positioning, establish an environmental data information set, and then jump to step S3;
[0018] Step S2.1.2: When the vehicle does not have map data information, data fusion is performed through the vehicle-mounted multi-source sensor data to assist in determining the lane-level high-precision positioning information, and then jump to step S3;
[0019] Step S3: through the confirmation and collection of the previous data of steps S1-S2, it is determined in real time whether the current position of the vehicle is in a high-risk area;
[0020] Further, the high-risk area refers to bridges, viaducts, tunnels, lanes with water or drop height differences on one or both sides, and curves; further, the curves are Class II high-risk areas, marked H2, and the rest of the high-risk areas are Class I, marked H1;
[0021] Further, if the current position of the vehicle is a high-risk area, then proceed to step S4, otherwise, proceed to step S7;
[0022] Step S4: Determine whether the current area is a high-risk area H1 through high-precision map information. If yes, jump to step S10; otherwise, jump to step S5;
[0023] Step S5: The alarm module prompts the vehicle to be in the H2 dangerous area through the vehicle positioning data and lane data, and prompts the vehicle to keep the lane, requiring the vehicle to drive in the lane and avoid crossing or pressing the lane; the gyroscope measures the vehicle deviation angle, and if a large angle abnormal deviation occurs suddenly, the processor can determine the lane deviation degree and the obstacle ahead according to the data of the multi-source sensors to control the vehicle body;
[0024] Step S6: Read the current speed data of the vehicle for determination. If the vehicle speed is less than 30 km / h, no vehicle speed reduction prompt is given. If the speed is greater than or equal to the speed limit required by the high-precision map, the vehicle is reminded to slow down. Enter step S3 to re-determine the current position of the vehicle.
[0025] Step S7: if it is determined according to step S3 that the vehicle is not currently in a high-risk area, the high-precision map information is read, and the distance to the nearest high-risk area on the route is measured, and the measured distance is compared with the high-risk area warning distance S. If it is less than or equal to S, the process proceeds to step S8, and the alarm module issues a warning; if the measured distance is greater than S, the process jumps to step S9, and the current state of the vehicle is determined;
[0026] Furthermore, the high-risk area warning distance S needs to be preset with a value in advance, and the value range can be set arbitrarily, and the preferred value range is between 100-500m;
[0027] Step S8: If the determination in step S7 is true, an early warning is issued through the alarm module;
[0028] Step S9: Read the fusion data of the vehicle's multi-source sensors, make a lane-level determination of whether the vehicle's current driving position is within the lane line, and determine whether the vehicle can continue to drive normally within the lane line without additional actions based on the driving trajectory and body posture. If the determination result is yes, jump to step 10, otherwise jump to step 13;
[0029] Step S10: Read the vehicle speed data in real time and monitor whether the steering wheel steering sensor has a sudden and large steering action;
[0030] Step S11: Determine whether the distance that the vehicle deviates from the lane line reaches a preset threshold value Qsh. If yes, jump to step S13, otherwise jump to step S12; further, the lane line deviation threshold value Qsh needs to be preset, and the Qsh is the corner radar of the lane line crossed by the vehicle as the target, and the vertical projection point of its spatial position is perpendicular to the lane line crossed, and the distance value of the perpendicular line is measured as Qsh; the preferred Qsh value is in the range of 20-30CM, and can also be manually set according to the vehicle model;
[0031] Step S12: Read the steering wheel steering sensor value to determine whether it reaches the threshold range. If so, proceed to step S14; otherwise, return to step S3 and repeat steps S3-S11 to monitor the vehicle deviation. Further, the threshold needs to be given a preset value, preferably in a range of greater than 60°, and can be reset manually.
[0032] Step S13: Read the vehicle multi-source sensor fusion data to make a judgment, and take deviation correction, avoidance, and emergency braking actions on the vehicle according to the current deviation degree and deviation trend of the vehicle;
[0033] Step S14: Read the multi-source sensor fusion data to determine whether there is an obstacle in the direction of the vehicle. If yes, jump to step S13; otherwise, jump to step 15;
[0034] Step S15: Through the comprehensive judgment of steps S9-S14, the current vehicle has experienced dangerous driving behaviors such as large lane deviation or sudden turn. The system predicts and takes emergency measures for the dangerous driving behaviors, and then alarms the current state of the vehicle through the alarm module, and prompts the driver to take over the vehicle actively.
[0035] Beneficial Effects
[0036] It can be used to distinguish dangerous sections of the route and provide early warning, intelligently identify and judge lane departure behaviors on different road sections and at different vehicle speeds, and actively identify and intervene in sudden high-risk driving behaviors. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0038] Figure 1 A schematic flow chart of a lane departure warning method based on high-precision maps and multi-source sensor fusion data provided by an embodiment of the present invention;
[0039] Figure 2 A schematic diagram of a lane departure warning device based on high-precision maps and multi-source sensor fusion data provided by an embodiment of the present invention; DETAILED DESCRIPTION
[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0041] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0042] The processor is used for processing and fusing data collected by multiple sensors, and is connected to the vehicle CAN bus to achieve active intervention and control of the vehicle.
[0043] The corner radar, laser radar, millimeter wave radar, and camera are used to collect lane environment information, and the processor performs data fusion to determine and identify the lane line data information, traffic sign information, and environmental data information in front of the vehicle.
[0044] The positioning module is used for collecting and processing vehicle satellite positioning data.
[0045] The storage module is used for storing and caching data information and can pre-store vehicle-mounted map data information.
[0046] The alarm module receives the processor data instruction and issues sound and light alarm information.
[0047] The communication module is used for accessing external devices and receiving information. The system does not rely on vehicle-mounted hardware devices and can establish connections with mobile phones and external terminal devices through the communication module to achieve access to high-precision maps and positioning data.
[0048] The steering wheel steering sensor and acceleration sensor / gyroscope are used to collect data on the vehicle's own speed, steering angle, and steering wheel inclination.
[0049] Furthermore, a lane departure warning method based on high-precision maps and multi-source sensor fusion data includes the following steps:
[0050] Step S1: Turn on the vehicle satellite positioning and receive positioning information; the vehicle positioning information can be based on the vehicle's own satellite positioning device, or the vehicle positioning can be achieved by connecting an external device through the communication module;
[0051] Step S2: Based on the vehicle satellite positioning information, determine whether the positioning accuracy is high-precision positioning and whether high-precision map assistance has been enabled. If yes, proceed to step S3; otherwise, proceed to step S2.1;
[0052] Step S2.1: In the absence of high-precision maps and high-precision positioning, further determine whether there is in-vehicle map assistance, and the in-vehicle map data can be connected to an external device through the communication module. If yes, proceed to step S2.1.1, otherwise, proceed to step S2.1.2;
[0053] Step S2.1.1: With the assistance of the on-board map, the multi-source sensor data of the vehicle is integrated to assist in determining the lane line position information, achieve lane-level high-precision positioning, establish an environmental data information set, and then jump to step S3;
[0054] Step S2.1.2: When the vehicle does not have map data information, data fusion is performed through the vehicle-mounted multi-source sensor data to assist in determining the lane-level high-precision positioning information, and then jump to step S3;
[0055] Step S3: through the confirmation and collection of the previous data of steps S1-S2, it is determined in real time whether the current position of the vehicle is in a high-risk area;
[0056] Further, the high-risk area refers to bridges, viaducts, tunnels, lanes with water or drop height differences on one or both sides, and curves; further, the curves are Class II high-risk areas, marked H2, and the rest of the high-risk areas are Class I, marked H1;
[0057] Further, if the current position of the vehicle is a high-risk area, then proceed to step S4, otherwise, proceed to step S7;
[0058] Step S4: Determine whether the current area is a high-risk area H1 through high-precision map information. If yes, jump to step S10; otherwise, jump to step S5;
[0059] Step S5: The alarm module prompts the vehicle to be in the H2 dangerous area through the vehicle positioning data and lane data, and prompts the vehicle to keep the lane, requiring the vehicle to drive in the lane and avoid crossing or pressing the lane; the gyroscope measures the vehicle deviation angle, and if a large angle abnormal deviation occurs suddenly, the processor can determine the lane deviation degree and the obstacle ahead according to the data of the multi-source sensors to control the vehicle body;
[0060] Step S6: Read the current speed data of the vehicle for determination. If the vehicle speed is less than 30 km / h, no vehicle speed reduction prompt is given. If the speed is greater than or equal to the speed limit required by the high-precision map, the vehicle is reminded to slow down. Enter step S3 to re-determine the current position of the vehicle.
[0061] Step S7: if it is determined according to step S3 that the vehicle is not currently in a high-risk area, the high-precision map information is read, and the distance to the nearest high-risk area on the route is measured, and the measured distance is compared with the high-risk area warning distance S. If it is less than or equal to S, the process proceeds to step S8, and the alarm module issues a warning; if the measured distance is greater than S, the process jumps to step S9, and the current state of the vehicle is determined;
[0062] Furthermore, the high-risk area warning distance S needs to be preset with a value in advance, and the value range can be set arbitrarily, and the preferred value range is between 100-500m;
[0063] Step S8: If the determination in step S7 is true, an early warning is issued through the alarm module;
[0064] Step S9: Read the fusion data of the vehicle's multi-source sensors, make a lane-level determination of whether the vehicle's current driving position is within the lane line, and determine whether the vehicle can continue to drive normally within the lane line without additional actions based on the driving trajectory and body posture. If the determination result is yes, jump to step 10, otherwise jump to step 13;
[0065] Step S10: Read the vehicle speed data in real time and monitor whether the steering wheel steering sensor has a sudden and large steering action;
[0066] Step S11: Determine whether the distance that the vehicle deviates from the lane line reaches a preset threshold value Qsh. If yes, jump to step S13, otherwise jump to step S12; further, the lane line deviation threshold value Qsh needs to be preset, and the Qsh is the corner radar of the lane line crossed by the vehicle as the target, and the vertical projection point of its spatial position is perpendicular to the lane line crossed, and the distance value of the perpendicular line is measured as Qsh; the preferred Qsh value is in the range of 20-30CM, and can also be manually set according to the vehicle model;
[0067] Step S12: Read the steering wheel steering sensor value to determine whether it reaches the threshold range. If so, proceed to step S14; otherwise, return to step S3 and repeat steps S3-S11 to monitor the vehicle deviation. Further, the threshold needs to be given a preset value, preferably in a range of greater than 60°, and can be reset manually.
[0068] Step S13: Read the vehicle multi-source sensor fusion data to make a judgment, and take deviation correction, avoidance, and emergency braking actions on the vehicle according to the current deviation degree and deviation trend of the vehicle;
[0069] Step S14: Read the multi-source sensor fusion data to determine whether there is an obstacle in the direction of the vehicle. If yes, jump to step S13; otherwise, jump to step 15;
[0070] Step S15: Through the comprehensive judgment of steps S9-S14, the current vehicle has experienced dangerous driving behaviors such as large lane deviation or sudden turn. The system predicts and takes emergency measures for the dangerous driving behaviors, and then alarms the current state of the vehicle through the alarm module, and prompts the driver to take over the vehicle actively.
[0071] It should be understood that the exemplary embodiments described herein are illustrative rather than restrictive. Although one or more embodiments of the present invention have been described in conjunction with the accompanying drawings, it should be understood by those skilled in the art that various changes in form and detail may be made without departing from the spirit and scope of the present invention as defined by the appended claims.
Claims
1. A lane departure warning method and device based on high-precision maps and multi-source sensor fusion data, characterized in that: include: The processor is used for processing and fusing data collected by multiple sensors, and is connected to the vehicle CAN bus to achieve active intervention and control of the vehicle; The corner radar, laser radar, millimeter wave radar, and camera are used to collect lane environment information, and the processor performs data fusion to determine and identify lane line data information, traffic sign information, and environmental data information in front of the vehicle; The positioning module is used for collecting and processing vehicle satellite positioning data; The storage module is used for storing and caching data information, and can pre-store vehicle map data information; The alarm module receives the processor data instruction and issues sound and light alarm information; The communication module is used for accessing external devices and receiving information. The system does not rely on the vehicle-mounted hardware equipment and can establish a connection with mobile phones and external terminal devices through the communication module to achieve access to high-precision maps and positioning data; The steering wheel steering sensor and acceleration sensor / gyroscope are used to collect data on the vehicle's own speed, steering angle, and steering wheel inclination.
2. A lane departure warning method and device based on high-precision map and multi-source sensor fusion data as claimed in claim 1, characterized in that The steps include: Step S1: Turn on the vehicle satellite positioning and receive positioning information; the vehicle positioning information can be based on the vehicle's own satellite positioning device, or the vehicle positioning can be achieved by connecting an external device through the communication module; Step S2: Based on the vehicle satellite positioning information, determine whether the positioning accuracy is high-precision positioning and whether high-precision map assistance has been enabled. If yes, proceed to step S3; otherwise, proceed to step S2.1; Step S2.1: In the absence of high-precision maps and high-precision positioning, further determine whether there is in-vehicle map assistance, and the in-vehicle map data can be connected to an external device through the communication module. If yes, proceed to step S2.1.1, otherwise, proceed to step S2.1.2; Step S2.1.1: With the assistance of the on-board map, the multi-source sensor data of the vehicle is integrated to assist in determining the lane line position information, achieve lane-level high-precision positioning, establish an environmental data information set, and then jump to step S3; Step S2.1.2: When the vehicle does not have map data information, data fusion is performed through the vehicle-mounted multi-source sensor data to assist in determining the lane-level high-precision positioning information, and then jump to step S3; Step S3: Through the confirmation and collection of the preliminary data of steps S1-S2, it is determined in real time whether the current position of the vehicle is in a high-risk area.
3. A lane departure warning method and device based on high-precision map and multi-source sensor fusion data as claimed in claim 2, characterized in that The high-risk area refers to bridges, viaducts, tunnels, lanes with water areas or drop height differences on one or both sides, and curves; further, the curves are Class II high-risk areas, marked H2, and the remaining high-risk areas are Class I, marked H1; further, if the current position of the vehicle is a high-risk area, proceed to step S4, otherwise, proceed to step S7.
4. The lane departure warning method and device based on high-precision map and multi-source sensor fusion data as claimed in claim 1, characterized in that: Step S4: Determine whether the current area is a high-risk area H1 through high-precision map information. If yes, jump to step S10; otherwise, jump to step S5; Step S5: The alarm module prompts the vehicle to be in the H2 dangerous area through the vehicle positioning data and lane data, and prompts the vehicle to keep the lane, requiring the vehicle to drive in the lane and avoid crossing or pressing the lane; the gyroscope measures the vehicle deviation angle, and if a large angle abnormal deviation occurs suddenly, the processor can determine the lane deviation degree and the obstacle ahead according to the data of the multi-source sensors to control the vehicle body; Step S6: Read the current speed data of the vehicle for determination. If the vehicle speed is less than 30 km / h, no vehicle speed reduction prompt is given. If the speed is greater than or equal to the speed limit required by the high-precision map, the vehicle is reminded to slow down. Enter step S3 to re-determine the current position of the vehicle. Step S7: If it is determined according to step S3 that the vehicle is not currently in a high-risk area, the high-precision map information is read, and the distance to the nearest high-risk area on the forward route is measured. The measured distance is compared with the high-risk area warning distance S. If it is less than or equal to S, the process proceeds to step S8, and the alarm module issues a warning; if the measured distance is greater than S, the process jumps to step S9 to determine the current body state of the vehicle.
5. A lane departure warning method and device based on high-precision map and multi-source sensor fusion data as claimed in claim 4, characterized in that The high-risk area warning distance S needs to be preset with a value in advance, and the value range can be set arbitrarily, and the preferred value range is between 100-500m.
6. The lane departure warning method and device based on high-precision map and multi-source sensor fusion data as claimed in claim 1, characterized in that: Step S8: If the determination in step S7 is true, an early warning is issued through the alarm module; Step S9: Read the fusion data of the vehicle's multi-source sensors, make a lane-level determination of whether the vehicle's current driving position is within the lane line, and determine whether the vehicle can continue to drive normally within the lane line without additional actions based on the driving trajectory and body posture. If the determination result is yes, jump to step 10, otherwise jump to step 13; Step S10: Read the vehicle speed data in real time and monitor whether the steering wheel steering sensor has a sudden and large steering action; Step S11: Determine whether the distance that the vehicle deviates from the lane line reaches a preset threshold value Qsh. If yes, jump to step S13, otherwise jump to step S12; further, the lane line deviation threshold value Qsh needs to be preset, and the Qsh is the corner radar of the lane line crossed by the vehicle as the target, and the vertical projection point of its spatial position is perpendicular to the lane line crossed, and the distance value of the perpendicular line is measured as Qsh; the preferred Qsh value is in the range of 20-30CM, and can also be manually set according to the vehicle model; Step S12: Read the steering wheel steering sensor value to determine whether it reaches the threshold range. If so, proceed to step S14; otherwise, return to step S3 and repeat steps S3-S11 to monitor the vehicle deviation. Further, the threshold needs to be given a preset value, preferably in a range of greater than 60°, and can be reset manually. Step S13: Read the vehicle multi-source sensor fusion data to make a judgment, and take deviation correction, avoidance, and emergency braking actions on the vehicle according to the current deviation degree and deviation trend of the vehicle; Step S14: Read the multi-source sensor fusion data to determine whether there is an obstacle in the direction of the vehicle. If yes, jump to step S13; otherwise, jump to step 15; Step S15: Through the comprehensive judgment of steps S9-S14, the current vehicle has experienced dangerous driving behaviors such as large lane deviation or sudden turn. The system predicts and takes emergency measures for the dangerous driving behaviors, and then alarms the current state of the vehicle through the alarm module, and prompts the driver to take over the vehicle actively.
Citation Information
Patent Citations
Road-vehicle fusion perception based early warning method against lane departure
CN110517521A
Multi-scene lane departure early warning system and method
CN110667574A
Lane departure early warning system and method based on high-precision map
CN113401136A
Lane departure early warning method and device, vehicle and medium
CN114348014A
Autonomous intelligent surveying and mapping system and method for lane line
CN115092159A
