A method and device for determining and warning of non-drivable lanes.

Abstract

This invention relates to a method and apparatus for determining and warning of non-drivable lanes. The method includes: acquiring positioning and perception information from multiple onboard sensors of a target vehicle; fusing the positioning information, perception information, and high-precision map data to obtain first lane attribute information of the target vehicle; parsing the perception information and comparing the parsed perception information with the high-precision map data to obtain second lane attribute information of the target vehicle; determining the consistency between the first and second lane attribute information, and judging whether the target vehicle is in a non-drivable lane based on the consistency result; and issuing a warning to the user based on the result of whether the target vehicle is in a non-drivable lane. This invention realizes a method for warning of non-drivable lanes in lane-level navigation engine products by fusing perception information, positioning information, and high-precision maps from multiple sensors.

Description

Technical Field

[0001] This invention belongs to the field of vehicle assisted driving technology, specifically relating to a method and device for determining and warning of non-drivable lanes. Background Technology

[0002] Humans can accurately determine key information such as drivable areas, road boundaries, lane lines, obstacles, and traffic rules using only their eyes (and some memory and knowledge), and control cars to drive safely accordingly. However, current human-designed sensors and post-processing algorithms cannot yet achieve the same performance.

[0003] Autonomous vehicles require high-precision maps and joint perception technologies to understand their surroundings. High-precision maps can provide vehicles with road information described in latitude and longitude, pre-collected by surveying vehicles, while all vehicles can broadcast real-time information about dynamic obstacles described in latitude and longitude to surrounding vehicles. The combination of these two technologies can greatly improve the safety of autonomous vehicles. Summary of the Invention

[0004] To address the issue of lane-level navigation engine products in determining and warning of non-drivable lanes, a first aspect of this invention provides a method for determining and warning of non-drivable lanes, comprising: acquiring positioning and perception information from multiple onboard sensors of a target vehicle, wherein the onboard sensors include a camera, a lidar, and a positioning and navigation device; fusing the positioning information, perception information, and high-precision map data to obtain first lane attribute information of the target vehicle; parsing the perception information and comparing the parsed perception information with the high-precision map data to obtain second lane attribute information of the target vehicle; determining the consistency between the first lane attribute information and the second lane attribute information, and determining whether the target vehicle is in a non-drivable lane based on the consistency result; and issuing a warning to the user based on the result of whether the target vehicle is in a non-drivable lane.

[0005] In some embodiments of the present invention, the step of fusing the positioning information, perception information and high-precision map data to obtain the first lane attribute information of the target vehicle includes: fusing the positioning results of the positioning and navigation device, camera and radar with the high-precision map data to obtain the first lane attribute information of the target vehicle.

[0006] In some embodiments of the present invention, the step of parsing the perceived information and comparing the parsed perceived information with high-precision map data to obtain the second lane attribute information of the target vehicle includes: acquiring an environmental image in front of the target vehicle through an ADAS camera, and identifying the third lane attribute information from the environmental image; acquiring a set of laser point clouds of the environment around the target vehicle through a LiDAR, and identifying the fourth lane attribute information from the set of laser point clouds; fusing the third lane attribute information and the fourth lane attribute information, and comparing the fused lane attribute information with high-precision map data to obtain the second lane attribute information of the target vehicle.

[0007] In some embodiments of the present invention, determining the consistency between the first lane attribute information and the second lane attribute information, and determining whether the target vehicle is on a non-drivable lane based on the consistency result, includes: if the first lane attribute information and the second lane attribute information are consistent, then determining whether the first lane attribute information is a non-drivable lane.

[0008] Furthermore, the first lane attribute information is obtained through fusion via the ADAS system, and the second lane attribute information is obtained through fusion via the IVI system.

[0009] In the above embodiments, issuing a warning to the user based on whether the target vehicle is in a non-drivable lane includes: if the target vehicle is in a non-drivable lane, issuing a voice reminder or image reminder to the user.

[0010] A second aspect of the present invention provides a device for determining and warning of non-drivable lanes, comprising: an acquisition module for acquiring positioning information and perception information from multiple on-board sensors of a target vehicle, wherein the on-board sensors include a camera, a lidar, and a positioning and navigation device; a fusion module for fusing the positioning information, perception information, and high-precision map data to obtain first lane attribute information of the target vehicle; a comparison module for parsing the perception information and comparing the parsed perception information with the high-precision map data to obtain second lane attribute information of the target vehicle; a judgment module for judging the consistency between the first lane attribute information and the second lane attribute information, and judging whether the target vehicle is in a non-drivable lane based on the consistency result; and a warning module for issuing a warning to a user based on the result of whether the target vehicle is in a non-drivable lane.

[0011] Furthermore, the comparison module includes: a first recognition unit, used to acquire an environmental image in front of the target vehicle through an ADAS camera, and to identify third lane attribute information from the environmental image; a second recognition unit, used to acquire a set of laser point clouds of the environment around the target vehicle through a lidar, and to identify fourth lane attribute information from the set of laser point clouds; and a comparison unit, used to fuse the third lane attribute information and the fourth lane attribute information, and to compare the fused lane attribute information with high-precision map data to obtain the second lane attribute information of the target vehicle.

[0012] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining and warning of non-drivable lanes provided in the first aspect of the present invention.

[0013] In a fourth aspect, the present invention provides a computer-readable medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the method for determining and warning of non-drivable lanes provided in the first aspect of the present invention.

[0014] The beneficial effects of this invention are: It can easily determine whether the lane in which the vehicle is traveling is a non-driving lane through onboard sensors and high-precision map data, and remind the user to leave the non-driving lane as soon as possible through the IVI system interface or sound to avoid driving hazards such as safety accidents and traffic violations. Attached Figure Description

[0015] Figure 1 This is a basic flowchart illustrating the method for determining and warning of non-drivable lanes in some embodiments of the present invention. Figure 2 This is a schematic diagram of the process of fusing first lane attribute information in some embodiments of the present invention; Figure 3 This is a schematic diagram of the process of fusing second lane attribute information in some embodiments of the present invention; Figure 4 This is a schematic diagram illustrating the specific process of determining and warning of non-drivable lanes in some embodiments of the present invention. Figure 5 This is a schematic diagram illustrating the application scenario effect of determining non-drivable lanes in some embodiments of the present invention. Figure 6 This is a schematic diagram of the structure of the non-drivable lane determination and early warning device in some embodiments of the present invention; Figure 7 This is a schematic diagram of the structure of an electronic device in some embodiments of the present invention. Detailed Implementation

[0016] The principles and features of the present invention are described below with reference to the accompanying drawings. The examples given are only for explaining the present invention and are not intended to limit the scope of the present invention.

[0017] refer to Figure 1 or Figure 4 In a first aspect, the present invention provides a method for determining and warning of non-drivable lanes, comprising: S100. acquiring positioning information and perception information from multiple on-board sensors of a target vehicle, wherein the on-board sensors include a camera, a lidar, and a positioning and navigation device; S200. fusing the positioning information, perception information, and high-precision map data to obtain first lane attribute information of the target vehicle; S300. parsing the perception information and comparing the parsed perception information with the high-precision map data to obtain second lane attribute information of the target vehicle; S400. determining the consistency between the first lane attribute information and the second lane attribute information, and determining whether the target vehicle is in a non-drivable lane based on the consistency result; S500. issuing a warning to a user based on the result of whether the target vehicle is in a non-drivable lane.

[0018] In step S100 of some embodiments of the present invention, positioning information and perception information of multiple on-board sensors of the target vehicle are acquired. The on-board sensors include cameras, lidar and positioning and navigation devices; for example, positioning information and perception information collected by sensors such as GNSS-RTK, IMU (Inertial Measurement Unit), ADAS (Advanced Driving Assistance System) cameras and radar.

[0019] refer to Figure 2 In step S200 of some embodiments of the present invention, fusing the positioning information, perception information and high-precision map data to obtain the first lane attribute information of the target vehicle includes: fusing the positioning results of the positioning and navigation device, camera and radar with the high-precision map data to obtain the first lane attribute information of the target vehicle.

[0020] Specifically, the fusion positioning of the vehicle intelligent driving system involves fusing the positioning results perceived by sensors such as GNSS-RTK, IMU, ADAS cameras, and radar with high-precision map data to obtain centimeter-level fusion positioning results. These results are then transmitted to the IVI (In-Vehicle Infotainment) system via a CAN network or Ethernet. The lane-level navigation engine product application installed in the IVI system can directly obtain these fusion positioning results. This achieves the transparent transmission process of lane-level positioning (centimeter-level) results.

[0021] refer to Figure 3 In step S300 of some embodiments of the present invention, parsing the perceived information and comparing the parsed perceived information with high-precision map data to obtain the second lane attribute information of the target vehicle includes: acquiring an environmental image in front of the target vehicle through an ADAS camera, and identifying the third lane attribute information from the environmental image; acquiring a set of laser point clouds of the environment around the target vehicle through a lidar, and identifying the fourth lane attribute information from the set of laser point clouds; fusing the third lane attribute information and the fourth lane attribute information, and comparing the fused lane attribute information with high-precision map data to obtain the second lane attribute information of the target vehicle.

[0022] Specifically, sensor perception and information recognition are mainly achieved through vehicle-mounted ADAS cameras and LiDAR.

[0023] ADAS cameras continuously capture images of the environment in front of the vehicle, and the intelligent driving system extracts special element information from the captured images for identification, such as directional arrows, bus lane markings, and tidal lane markings.

[0024] LiDAR continuously scans the area in front of and around the vehicle, generating a set of laser point clouds. The intelligent driving system extracts and identifies special elements from these point clouds, such as directional arrows, bus lane markings, and reversible lane markings. Based on these identified lane elements, the system assigns the most likely lane attribute to each lane, such as bus lane, reversible lane and its direction of travel, or emergency lane. This most likely lane attribute information is then sent to the IVI system. The ADAS camera and LiDAR sensors complement and verify each other's perception and identification of road elements, providing information redundancy to improve the recognition rate.

[0025] In step S400 of some embodiments of the present invention, determining the consistency between the first lane attribute information and the second lane attribute information, and determining whether the target vehicle is on a non-drivable lane based on the consistency result includes: if the first lane attribute information and the second lane attribute information are consistent, then determining whether the first lane attribute information is a non-drivable lane.

[0026] Specifically, the IVI system determines the lane where the vehicle is currently located by using the fusion positioning results transmitted by the intelligent driving system. Then, it compares the results with high-precision map data to obtain the attribute information of the current lane. This attribute information is then compared with the most likely lane attributes transmitted by the intelligent driving system for information verification, and finally, it determines whether the lane the vehicle is in is a non-drivable lane.

[0027] Each lane in a high-precision map has its own attribute information, as follows: 1. Straight lane 2. Left turn lane 3. Right turn lane 4. Left turn and straight lane 5. Right turn and straight lane 6. Left turn and U-turn lane 7. Dedicated U-turn lane 8. Dedicated bus lane 9. Tidal flow lane 10. Emergency lane 11. Emergency stopping lane 12. Manual toll lane at toll station 13. ETC lane at toll station 14. Non-motorized vehicle lane 15. Emergency escape lane, etc.

[0028] refer to Figure 5 Lane-level navigation engine products warn users of non-drivable lanes by displaying the warning message on the IVI interface after the IVI system issues it, reminding the user to leave the non-drivable lane as soon as possible. This is a direct application scenario of non-drivable lane determination. The IVI system interface highlights the non-drivable lane in red to remind the user to leave the lane immediately, displays the message "Leave now," and simultaneously provides a voice prompt: "You have entered a non-drivable lane. To avoid a traffic violation, please leave as soon as possible." Once the lane-level navigation engine determines that the vehicle is no longer in the non-drivable lane, the warning message is removed, and normal navigation resumes.

[0029] refer to Figure 4 In one embodiment of the method for determining and warning of non-drivable lanes provided by the present invention, the following steps are included: Step 1: The vehicle sensors send positioning and perception information to the intelligent driving system. The system compares the perception information with high-precision map data to obtain positioning information from the high-precision map. The sensor positioning information and high-precision map positioning information are then fused using an algorithm to obtain lane-level (centimeter-level) fused positioning information, which is then sent to the IVI system.

[0030] Step 2: The vehicle sensors send perception information to the intelligent driving system. The intelligent driving system analyzes and recognizes the perception information using algorithms, and compares it with high-precision map data to obtain the most likely lane attribute of the vehicle's lane.

[0031] Step 3: The IVI system compares the fusion positioning results sent by the intelligent driving system with the high-precision map data of the IVI system to obtain the most likely lane attribute of the lane where the vehicle is located.

[0032] Step 4: Compare the most likely lane attribute of the vehicle's lane on the intelligent driving terminal with the most likely lane attribute on the IVI terminal. If they do not match, an error message will be displayed. If they match, proceed to Step 5.

[0033] Step 5: Determine whether the most likely lane attribute obtained in Step 4 is a non-drivable lane. If it is, end the current round of judgment; otherwise, proceed to Step 6.

[0034] Step Six: The IVI system sends a warning to the user through visual and audio information, prompting the user to leave the current lane as soon as possible. The warning ends when the program reaches Step Five and determines that the warning is not valid.

[0035] Example 2 refer to Figure 6 In a second aspect, the present invention provides a device 1 for determining and warning of non-drivable lanes, comprising: an acquisition module 11 for acquiring positioning information and perception information from multiple on-board sensors of a target vehicle, wherein the on-board sensors include a camera, a lidar, and a positioning and navigation device; a fusion module 12 for fusing the positioning information, perception information, and high-precision map data to obtain first lane attribute information of the target vehicle; a comparison module 13 for parsing the perception information and comparing the parsed perception information with the high-precision map data to obtain second lane attribute information of the target vehicle; a judgment module 14 for judging the consistency between the first lane attribute information and the second lane attribute information, and judging whether the target vehicle is in a non-drivable lane based on the consistency result; and a warning module 15 for issuing a warning to the user based on the result of whether the target vehicle is in a non-drivable lane.

[0036] Furthermore, the comparison module 13 includes: a first identification unit, used to acquire an environmental image in front of the target vehicle through an ADAS camera, and identify third lane attribute information from the environmental image; a second identification unit, used to acquire a set of laser point clouds of the environment around the target vehicle through a lidar, and identify fourth lane attribute information from the set of laser point clouds; and a comparison unit, used to fuse the third lane attribute information and the fourth lane attribute information, and compare the fused lane attribute information with high-precision map data to obtain the second lane attribute information of the target vehicle.

[0037] Example 3 refer to Figure 7 In a third aspect, the present invention provides an electronic device comprising: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method for determining and warning of non-drivable lanes of the present invention in the first aspect.

[0038] Electronic device 500 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via bus 504. An input / output (I / O) interface 505 is also connected to bus 504.

[0039] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, hard disks; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 7 Each box shown can represent a device or multiple devices as needed.

[0040] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 509, or installed from a storage device 508, or installed from a ROM 502. When the computer program is executed by a processing device 501, it performs the functions defined in the methods of embodiments of this disclosure. It should be noted that the computer-readable medium described in embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Program code contained on a computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0041] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more computer programs, which, when executed by the electronic device, cause the electronic device to: Computer program code for performing the operations of embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages—such as Java, Smalltalk, C++, and Python—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0042] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0043] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for determining and issuing early warnings of non-drivable lanes, characterized in that, include: The system acquires positioning and perception information from multiple onboard sensors of the target vehicle, including cameras, lidar, and positioning and navigation devices. The positioning information, perception information, and high-precision map data are fused to obtain the first lane attribute information of the target vehicle; The perceived information is analyzed, and the analyzed perceived information is compared with high-precision map data to obtain the second lane attribute information of the target vehicle; The step of parsing the perceived information and comparing the parsed perceived information with high-precision map data to obtain the second lane attribute information of the target vehicle includes: The ADAS camera acquires an environmental image of the area in front of the target vehicle, and identifies the third lane attribute information from the environmental image. The laser point cloud set of the environment around the target vehicle is obtained by LiDAR, and the attribute information of the fourth lane is identified from the laser point cloud set. The third lane attribute information and the fourth lane attribute information are fused together, and the fused lane attribute information is compared with high-precision map data to obtain the second lane attribute information of the target vehicle. Determine the consistency between the first lane attribute information and the second lane attribute information, and determine whether the target vehicle is in a non-drivable lane based on the consistency result; The system issues a warning to the user based on whether the target vehicle is in a non-driving lane.

2. The method for determining and warning of non-drivable lanes according to claim 1, characterized in that, The step of fusing the positioning information, perception information, and high-precision map data to obtain the first lane attribute information of the target vehicle includes: The positioning results from navigation devices, cameras, and radar are fused with high-precision map data to obtain the first lane attribute information of the target vehicle.

3. The method for determining and warning of non-drivable lanes according to claim 1, characterized in that, The step of determining the consistency between the first lane attribute information and the second lane attribute information, and determining whether the target vehicle is on a non-drivable lane based on the consistency result, includes: If the attribute information of the first lane is the same as that of the second lane, then determine whether the attribute information of the first lane is a non-drivable lane.

4. The method for determining and warning of non-drivable lanes according to claim 3, characterized in that, The first lane attribute information is obtained by fusing the ADAS system, and the second lane attribute information is obtained by fusing the IVI system.

5. The method for determining and warning of non-drivable lanes according to any one of claims 1 to 4, characterized in that, The method of issuing a warning to the user based on whether the target vehicle is in a non-drivable lane includes: If the target vehicle is in a lane that is not allowed to drive, a voice or image reminder will be issued to the user.

6. A device for determining and warning of non-drivable lanes, characterized in that, include: The acquisition module is used to acquire the positioning and perception information of multiple on-board sensors of the target vehicle, including cameras, lidar, and positioning and navigation devices. The fusion module is used to fuse the positioning information, perception information and high-precision map data to obtain the first lane attribute information of the target vehicle; The comparison module is used to parse the perceived information and compare the parsed perceived information with high-precision map data to obtain the second lane attribute information of the target vehicle. The comparison module includes: The first identification unit is used to acquire an environmental image in front of the target vehicle through an ADAS camera and identify the third lane attribute information from the environmental image. The second identification unit is used to acquire a set of laser point clouds of the environment around the target vehicle through lidar, and to identify the fourth lane attribute information from the set of laser point clouds. The comparison unit is used to fuse the third lane attribute information and the fourth lane attribute information, and compare the fused lane attribute information with high-precision map data to obtain the second lane attribute information of the target vehicle. The judgment module is used to determine the consistency between the first lane attribute information and the second lane attribute information, and to determine whether the target vehicle is in a non-drivable lane based on the consistency result. The warning module is used to issue warnings to users based on whether the target vehicle is in a non-drivable lane.

7. An electronic device, comprising: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method for determining and warning of non-drivable lanes as described in any one of claims 1 to 5.

8. A computer-readable medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the method for determining and warning of non-drivable lanes as described in any one of claims 1 to 5.

Images