Method for judging drivable area applied to intelligent driving, intelligent driving system and intelligent automobile

Abstract

The embodiment of the invention discloses a drivable area judgment method applied to intelligent driving, an intelligent driving system and an intelligent automobile, and the method comprises the steps: determining a current drivable area according to the environment information around the position where the automobile is located, querying a historical drivable area library according to the position where the automobile is located, obtaining the information of a corresponding historical drivable area, and determining the drivable area according to the information of the historical drivable area. And superposing the current drivable area and the historical drivable area to obtain the drivable area of this time.

Description

[0001] The present application is a divisional application, the original application number is 202010726575.7, the original application date is July 25, 2020, and the entire contents of the original application are incorporated herein by reference. TECHNICAL FIELD

[0002] The present application relates to the field of intelligent vehicles, in particular to a drivable area determination method applied to intelligent driving, an intelligent driving system and an intelligent vehicle. BACKGROUND

[0003] With the development of economy, the number of vehicles in use is rapidly increasing, and vehicle technology is increasingly integrated with computer technology. In recent years, intelligent vehicles have become a new trend in vehicle development, and more and more vehicles have adopted driver assistance, automated driving or intelligent network driving systems. Such systems intelligently detect obstacles and perceive the surrounding environment through vehicle-mounted image acquisition devices and vehicle-mounted sensors during driving, and use vehicle-mounted computing platforms (e.g., mobile data centers (MDC)) to determine the driving path of the vehicle and control the driving state of the vehicle.

[0004] Automated driving technology has broad application prospects and important research significance. Automated driving detects roads and obstacles through perception devices and autonomously performs driving operations, which can improve driving safety, reduce traffic accident rates and reduce personnel and economic losses. At the same time, automated driving can also cooperate with intelligent transportation systems to more reasonably allocate road resources and alleviate urban congestion. The current automated driving technology is still in the research and testing stage, and drivable area detection is an essential part of advanced driver assistance and automated driving.

[0005] The drivable area detection method is a method for determining the drivable area based on the input of the current perception device. Existing technologies aim to instantly determine the road surface and identify the drivable area through machine learning. However, automated driving based on current drivable area detection technology only plans and controls the drivable path based on the detected current drivable area, and the accuracy of path planning is limited. SUMMARY

[0006] The present application provides a drivable area determination method applied to intelligent driving, an intelligent driving system and an intelligent vehicle, which solves the problem that the existing drivable area determination is instantaneous and has limited accuracy and cannot be self-learned.

[0007] In a first aspect, embodiments of the present application provide a drivable area determination method applied to intelligent driving, comprising:

[0008] An intelligent driving system acquires environmental information around a position where a vehicle is located, and determines a current drivable area;

[0009] The intelligent driving system queries a historical drivable area library according to the position where the vehicle is located, and obtains information of a corresponding historical drivable area;

[0010] The intelligent driving system superimposes the current drivable area and the historical drivable area, and obtains a drivable area for this time.

[0011] Embodiments of the present application persist the determination result of the previous drivable area to the historical drivable area library, and apply it to the subsequent drivable area determination process, so that the intelligent driving system can select a driving path for this time from the current drivable area, to output the drivable area for this time including the optimal path.

[0012] The intelligent driving system determines whether the superimposed drivable area contains a drivable lane satisfying a first length, and if so, takes the drivable lane satisfying the first length as the drivable area for this time. When the superimposed drivable area does not contain a drivable lane satisfying the first length, the current drivable area is taken as the drivable area for this time.

[0013] Embodiments of the present application take the historical drivable area of the position where the vehicle is located as a factor for determining the drivable area for this time. In a possible implementation, only when the superimposed drivable area contains a drivable lane satisfying the first length, the superimposed drivable area is output as the drivable area for this time. At this time, the drivable lane satisfying the first length can be taken as the optimal path, so that the accuracy of the output drivable area for this time is higher.

[0014] In a possible implementation, embodiments of the present application superimpose the current drivable area and the historical drivable area to obtain the drivable area for this time, which can be:

[0015] When the superimposed drivable area contains a drivable lane satisfying the first length, the superimposed drivable area is taken as the drivable area for this time.

[0016] if the current drivable area and the drivable area obtained by superimposing the previous k historical drivable areas do not contain a drivable lane satisfying the first length, it is determined whether the current drivable area and the drivable area obtained by superimposing the last historical drivable area contain a drivable lane satisfying the first length, and if so, the drivable area obtained by superimposing the current drivable area and the last historical drivable area is taken as the drivable area of this time;

[0017] wherein k is a positive integer greater than or equal to 2.

[0018] The two methods of superimposing the drivable area can preferably superimpose the current drivable area and the previous k historical drivable areas. If there is a drivable lane satisfying the first length after superimposing the current drivable area and the previous k historical drivable areas, it indicates that the optimal path is combined with the previous k historical drivable areas, and the path is also a preferred path with a high probability in the previous k driving processes. Sub-optimally, if there is no drivable lane satisfying the first length after superimposing the current drivable area and the previous k historical drivable areas, the previous drivable area and the last historical drivable area can be superimposed. If there is a drivable lane satisfying the first length after superimposition, it indicates that the drivable lane satisfying the first length is a drivable lane in this time and the last driving process.

[0019] In a possible implementation, when the current drivable area includes a drivable lane satisfying the first length, the current drivable area is updated to the historical drivable area library.

[0020] In another possible implementation, the superimposed drivable area can be used to update the historical drivable area library by the embodiments of the application. For example:

[0021] When the current drivable area and the drivable area obtained by superimposing the previous k historical drivable areas contain a drivable lane satisfying the first length, the superimposed drivable area is output as the drivable area of this time, and is recorded to the historical drivable area library;

[0022] When the current drivable area and the drivable area obtained by superimposing the previous k historical drivable areas do not contain a drivable lane satisfying the first length, and the current drivable area and the drivable area obtained by superimposing the last historical drivable area contain a drivable lane satisfying the first length, the drivable area obtained by superimposing the current drivable area and the last historical drivable area is output as the drivable area of this time, and is recorded to the historical drivable area library;

[0023] When the drivable area obtained by the superposition does not contain a drivable lane satisfying the first length, and the current drivable area contains a drivable lane satisfying the first length, the current drivable area is output as the drivable area of the current time, and the record is recorded in the historical drivable area library.

[0024] When the number of records corresponding to the positioning data in the historical drivable area library exceeds k, the record with the longest storage time is deleted.

[0025] By the above updating method, the timeliness of the data recorded in the historical drivable area library is improved, and the old data stored in the historical drivable area library is avoided, which affects the determination result of the subsequent drivable area.

[0026] In an example, the first length represents the range perceived by the intelligent driving system, and the drivable lane satisfying the first length represents a lane that is a drivable area within the range perceived by the intelligent driving system.

[0027] In a possible implementation, the historical drivable area library records information of the historical drivable area, including historical drivable area records, positioning data, and road feature points. The intelligent driving system stores the information of the historical drivable area with a second length as a granularity. The second length can be an experience value, such as 1 meter or 10 meters, etc. The smaller the second length is, the more information of the historical drivable area needs to be stored, and more storage space is occupied.

[0028] In an example, the information of the historical drivable area further includes a label number. Each historical drivable area record corresponds to a label number, and the label number corresponds to the positioning data. The intelligent driving system can query the label number of the positioning data according to the positioning data of the location of the vehicle, and output all information of the historical drivable area matching the label number.

[0029] It can be understood that the output historical drivable area is greater than or equal to the perception range of the intelligent driving system, that is, the output historical drivable area covers the perception range in the forward direction of the vehicle. At this time, when the second length is smaller than the radius of the perception range of the intelligent driving system, the intelligent driving system obtains the historical drivable area in the forward direction of the vehicle with a length greater than or equal to the perception range of the vehicle, that is, obtains the information of the historical drivable area corresponding to multiple label numbers in the forward direction, so that the range represented by the set of the historical drivable areas corresponding to multiple label numbers is greater than or equal to the perception range in the forward direction of the vehicle.

[0030] In a possible implementation, after the historical drivable area library is queried to obtain the information of the corresponding historical drivable area, the method further includes: the intelligent driving system registers the current drivable area and the historical drivable area according to the road feature points of the position where the vehicle is located, so that the road feature points of the current drivable area coincide with the road feature points included in the information of the queried historical drivable area on the map.

[0031] In a second aspect, the embodiments of the present application provide an intelligent driving system, including:

[0032] a detection module configured to acquire environmental information around a position where a vehicle is located, and determine a current drivable area;

[0033] a query module configured to query a historical drivable area library according to the position where the vehicle is located, and obtain information of a corresponding historical drivable area;

[0034] a fusion module configured to superimpose the current drivable area and the historical drivable area, and obtain a drivable area of this time.

[0035] In a possible implementation, the fusion module is specifically configured to determine whether the superimposed drivable area contains a drivable lane that meets a first length, and if so, take the drivable lane that meets the first length as the drivable area of this time.

[0036] The fusion module is specifically configured to, when the superimposed drivable area does not contain the drivable lane that meets the first length, take the current drivable area as the drivable area of this time.

[0037] The fusion module is specifically configured to, when the current drivable area and the superimposed drivable area of the previous k historical drivable areas contain the drivable lane that meets the first length, take the superimposed drivable area as the drivable area of this time.

[0038] The fusion module is specifically configured to, when the current drivable area and the superimposed drivable area of the previous k historical drivable areas do not contain the drivable lane that meets the first length, determine whether the current drivable area and the superimposed drivable area of the last historical drivable area contain the drivable lane that meets the first length, and if so, take the current drivable area and the superimposed drivable area of the last historical drivable area as the drivable area of this time, where k is a positive integer greater than or equal to 2.

[0039] The fusion module is specifically configured to, when the current drivable area includes the drivable lane that meets the first length, update the current drivable area to the historical drivable area library.

[0040] The information of the historical drivable area includes a historical drivable area record, positioning data, and road feature points.

[0041] The fusion module is specifically configured to register the current drivable area and the historical drivable area according to the road feature points of the location of the vehicle, so that the road feature points of the current drivable area coincide with the road feature points included in the information of the historical drivable area obtained by the query on a map.

[0042] In a third aspect, the embodiments of the present application further provide an intelligent vehicle, comprising a processor, a memory and a perception device,

[0043] The memory is configured to store a historical drivable area library.

[0044] The perception device is configured to obtain environmental information around a location of the vehicle.

[0045] The processor is configured to execute instructions to implement the method described in any of the specific implementation manners of the first aspect.

[0046] In a fourth aspect, the embodiments of the present application provide a drivable area determination device, comprising a processor, a communication interface and a memory; the memory is configured to store instructions, the processor is configured to execute the instructions, and the communication interface is configured to receive or send data; wherein the processor executes the instructions to implement the method described in any of the specific implementation manners of the first aspect.

[0047] In a fifth aspect, the embodiments of the present application provide a non-transient computer storage medium, which stores a computer program; when the computer program is executed by a processor, the method described in the first aspect or any of the specific implementation manners of the first aspect is implemented.

[0048] On the basis of the implementation manners of the above aspects, the embodiments of the present application can be further combined to provide more implementation manners.

[0049] The embodiment of the present application discloses a drivable area determination method applied to intelligent driving. According to the environmental information around the position of a vehicle, the current drivable area is determined. According to the position of the vehicle, the historical drivable area library is queried to obtain the information of the corresponding historical drivable area. The current drivable area and the historical drivable area are superimposed to obtain the drivable area of this time. The embodiment of the present application provides a method for determining the drivable area of this time based on the information of the historical drivable area. The stored historical drivable area information is fused with the current drivable area to output the optimal drivable area. Further, the embodiment of the present application can combine road feature point detection to improve the accuracy of fusion. The embodiment of the present application can also update the historical drivable area to update the road condition changes to the historical record in time. Further, the embodiment of the present application can also set a historical information aging mechanism. After updating the information of the drivable area of this time to the historical drivable area library, at least one old data is discarded, so that the algorithm can eliminate the influence of outdated information and avoid interference from individual abnormal values. BRIEF DESCRIPTION OF DRAWINGS

[0050] Figure 1 Fig. 1 is a hardware structure schematic diagram of a vehicle-mounted computing system provided by the embodiment of the present application;

[0051] Figure 2 Fig. 2 is a logic structure schematic diagram of an intelligent driving system provided by the embodiment of the present application;

[0052] Figure 3 Fig. 3 is a time sequence flow schematic diagram of drivable area fusion determination provided by the embodiment of the present application;

[0053] Figure 4 Fig. 4 is a fusion schematic diagram of the current drivable area and the historical drivable area provided by the embodiment of the present application;

[0054] Figure 5 Fig. 5 is a perception device perception range schematic diagram provided by the embodiment of the present application.

[0055] Figure 6 Fig. 6 is a drivable area determination method flow schematic diagram applied to intelligent driving provided by the embodiment of the present application;

[0056] Figure 7 Fig. 7 is another drivable area determination method flow schematic diagram applied to intelligent driving provided by the embodiment of the present application;

[0057] Figure 8 Fig. 8 is a drivable area determination device schematic diagram provided by the embodiment of the present application. DETAILED DESCRIPTION

[0058] The embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0059] The embodiments of the present application are applied to the field of smart cars, such as Figure 1 FIG2 is a schematic diagram of the hardware structure of an on-vehicle computing system applicable to an embodiment of the present application. The on-vehicle computing system may include an on-vehicle processing system 101 and devices / components / networks directly or indirectly connected to the on-vehicle processing system 101.

[0060] See also Figure 1 The on-board processing system 101 includes a processor 103 and a system memory 135. The processor 103 is connected to other components / interfaces on the on-board processing system 101 via a system bus 105. The processor 103 can be one or more processors, each of which can include one or more processor cores. The on-board processing system 101 is connected to other components on the vehicle, such as a display 109, an interactive device 117, a multimedia device 121, a positioning device 123, and a perception device 153 (a camera and various sensors, etc.) through various interfaces (e.g., an adapter 107, an I / O interface 115, a USB port 125, etc.). The on-board processing system 101 is connected to an external network 127 via a network interface 129, and exchanges information with a server 149 via the external network 127. The network interface 129 can send and / or receive communication signals.

[0061] In the vehicle processing system 101, the system bus 105 is coupled to an input / output (I / O) bus 113 via a bus bridge 111. The I / O interface 115 communicates with various I / O devices, such as an interactive device 117 (e.g., a keyboard, mouse, touch screen, etc.), a multimedia device 121 (e.g., a compact disc read-only memory (CD-ROM), a multimedia interface, etc.), a positioning device 123, a universal serial bus (USB) port 125, and a sensing device 153 (e.g., a camera capable of capturing still and moving digital video images).

[0062] The interactive device 117 is used to implement message interaction between the smart car and the driver. The driver can select the driving mode and driving style model of the smart car through the interactive device 117. In one possible implementation, the interactive device 117 can be integrated with the display 109.

[0063] The processor 103 can be any conventional processor, including a reduced instruction set computer (RISC) processor, a complex instruction set computer (CISC) processor, or a combination of the above. Alternatively, the processor can be a special purpose device such as an application specific integrated circuit (ASIC). Alternatively, the processor 103 can be a neural network processor or a combination of a neural network processor and a conventional processor as described above. Alternatively, the processor 103 can include a main controller (also referred to as a central controller) and an advanced driver assistance system controller. The main controller is the control center of the computer system. The advanced driver assistance system controller is used to control the route of autonomous driving or assisted autonomous driving, etc.

[0064] The display 109 can be any one or more display devices installed in the vehicle. For example, the display 109 can include a head-up display (HUD), an instrument panel, and a display for passengers, etc.

[0065] The in-vehicle processing system 101 can communicate with a server 149 through a network interface 129. The network interface 129 can be a hardware network interface such as a network card. The network 127 can be an external network such as the Internet, or an internal network such as an Ethernet or a virtual private network (VPN). Alternatively, the network 127 can also be a wireless network such as a WiFi network, a cellular network, etc.

[0066] The memory interface 131 is coupled to the system bus 105. The memory interface is connected to the memory.

[0067] The system memory 135 is coupled to the system bus 105. The data running in the system memory 135 can include an operating system 137 and an intelligent driving system 143.

[0068] The operating system 137 includes a shell 139 and a kernel 141. The shell 139 is an interface between the user and the kernel of the operating system. The shell is the outermost layer of the operating system. The shell manages the interaction between the user and the operating system, waits for the user's input, interprets the user's input to the operating system, and processes various outputs of the operating system.

[0069] The kernel 141 is composed of those parts of the operating system that manage memory, files, peripherals, and system resources. Directly interacting with the hardware, the operating system kernel typically runs processes and provides inter-process communication, provides CPU time slicing management, interrupts, memory management, and I / O management, among other things.

[0070] The intelligent driving system 143 includes programs related to controlling automatic driving of the vehicle. For example, programs for processing the information about the environment around the vehicle obtained by the on-board device, such as programs for implementing the drivable path determination method provided by the embodiments of the present application. For another example, programs for controlling the route or speed of the automatic driving vehicle, programs for controlling the interaction between the automatic driving vehicle and other automatic driving vehicles on the road, and the like. The intelligent driving system 143 can also exist on the system of the server 149. In an embodiment, when the intelligent driving system 143 needs to be executed, the on-board processing system 101 can download the installation program of the intelligent driving system 143 from the server 149.

[0071] The perception device 153 is associated with the on-board processing system 101. The perception device 153 is used to detect the environment around the vehicle. For example, the perception device 153 can detect animals, other vehicles, obstacles, crosswalks, lane lines, and the like around the vehicle, and further detect the environment around the above-mentioned animals, vehicles, obstacles, and crosswalks, such as weather conditions, brightness of the surrounding environment, and the like. The perception device can include a camera, an infrared sensor, a chemical detector, a microphone, and the like. The perception device 153 can also include a speed sensor for measuring the speed information (such as speed, acceleration, and the like) of the vehicle (i.e., the vehicle on which the on-board computing system is located); an angle sensor for measuring the direction information of the vehicle and the relative angle between the vehicle and the objects / objects around the vehicle. The perception device 153 can also include a laser radar sensor for detecting the reflection signal of the laser signal sent by the laser radar, thereby obtaining the laser point cloud. The laser radar can be installed above the vehicle for sending laser signals. Figure 1 The perception device 153 can also include a laser radar sensor for detecting the reflection signal of the laser signal sent by the laser radar, thereby obtaining the laser point cloud. The laser radar can be installed above the vehicle for sending laser signals.

[0072] The positioning device 123 includes a global positioning system (GPS), an inertial navigation system (INS), and the like for determining the position of the vehicle.

[0073] In some embodiments of the present application, the processor 103 executes various instructions to implement various functions of the intelligent driving system 143. Specifically, for example, Figure 2As shown, the embodiment of the present application provides a structure example of the intelligent driving system 143, which can include a detection module 1431, a query module 1432, a fusion module 1433, and a vehicle control module 1434.

[0074] The detection module 1431 is configured to acquire environmental information around a position where a vehicle is located, and determine a current drivable area. Specifically, the detection module 1431 determines the environmental information around the vehicle through the positioning device 123 and the perception device 153, including information of obstacles (such as the position and size of obstacles, including but not limited to the position, size, pose and speed of real objects such as people, vehicles, roadblocks, etc.) in the surrounding area of the vehicle, and lane information, etc. The detection module 1431 determines the current drivable area according to the environmental information around the vehicle.

[0075] The query module 1432 is configured to query a historical drivable area library according to the position where the vehicle is located, and obtain information of a corresponding historical drivable area. The historical drivable area library records drivable paths of the vehicle when passing through the position where the vehicle is located in history.

[0076] The fusion module 1433 is configured to superimpose the current drivable area and the historical drivable area, and obtain a drivable area of this time.

[0077] The fusion module 1433 is specifically configured to determine whether the superimposed drivable area contains a drivable lane satisfying a first length, and if so, take the drivable lane satisfying the first length as the drivable area of this time.

[0078] Further, the fusion module 1433 notifies the vehicle control module 1434 to control the driving path of the vehicle according to the output drivable area of this time.

[0079] The storage 133 is configured to store historical drivable area information and a map (such as a high-definition map), and update the stored content. The detection module 1431 acquires information of a map in a certain area around the vehicle from the storage 133 according to the current position of the vehicle. The information of the map includes road identifiers in the certain area around the vehicle, such as road lines, lane lines, and stop lines. The detection module 1431 interacts with the perception device 153 to acquire obstacle conditions and road identifiers in the surrounding area of the vehicle, and performs registration operations.

[0080] The storage 133 can be a storage dedicated to storing maps, or a general storage. The embodiment of the present application does not limit this.

[0081] The above modules (the detection module 1431, the query module 1432, the fusion module 1433, and the vehicle control module 1434) can be implemented by software and / or hardware. Any one or more of the modules can be independently set or integrated together. The embodiments of the present application do not make specific limitations in this regard. In one example, any one or more of the modules can be a logical functional module in the main controller or the ADAS controller.

[0082] The vehicle-mounted computing system can be located entirely on the vehicle, or part of the processing logic can be located in a cloud network connected to the vehicle through a network. For example, the vehicle-mounted processing system 101 can be located away from the autonomous vehicle and can communicate wirelessly with the autonomous vehicle. In other aspects, some of the processes described herein are performed on a processor disposed within the autonomous vehicle, and others are performed by a remote processor. For example, all or part of the functions of the query module 1432 are implemented on a server deployed in the cloud.

[0083] It should be noted that, Figure 1 The computer system shown is only an example, and does not limit the computer system to which the embodiments of the present application can be applied.

[0084] The drivable area in the embodiments of the present application can include structured road surface, semi-structured road surface, unstructured road surface, and other road surface areas that can be used for intelligent vehicle driving. The structured road surface generally has road edge lines, and the road surface structure is single, such as city trunk roads, expressways, national highways, and provincial highways. The structure layer of this road surface performs certain standards, and the color and material of the surface layer are uniform. The semi-structured road surface refers to a generally non-standardized road surface, and the color and material of the road surface surface layer differ greatly, such as parking lots, squares, and some branch roads. The unstructured road surface has no structure layer and is a natural road scene. Autonomous driving needs to achieve path planning, and therefore must achieve detection of the drivable area. In an autonomous driving system based on computer vision, the drivable area is generally detected based on image segmentation technology. For city vehicles, autonomous driving solves the detection and identification of structured road surfaces and unstructured road surfaces. For off-road unmanned vehicles, the detection of unstructured road surfaces needs to be solved.

[0085] There are many different detection methods for different environments. The basic methods include obtaining the basic structural features of the road surface based on the road surface color, the road model, and the road surface texture features, and further obtaining the potential information such as the road edge line and the basic direction of the road (straight, left turn, right turn, left sharp turn, and right sharp turn) through these features. These features can be extracted through traditional segmentation extraction methods or machine learning methods.

[0086] The detection of the drivable area is mainly used to provide path planning assistance for automatic driving. The detection can be a detection of the entire road surface, or only part of the road information can be extracted, such as the road direction in a certain area in front, or the road midpoint or lane line, etc., as long as the road path planning and obstacle avoidance can be realized in combination with the high-precision map, and the complete drivable area of the road surface does not have to be extracted. For semi-structured or unstructured road surfaces without clear road markings, there is no lane division on the road surface at this time, and the intelligent driving system 143 can further perform an analysis function to divide the lanes in the detection range, and apply the drivable area detection result after dividing the lanes to the subsequent processing process.

[0087] For repeated roads in the automatic driving scene, especially for commuting road lines, the prior art cannot make a prediction of the road conditions of the historical area after multiple passes, like human drivers. For example, which section of the lane has potholes, which section is a complex intersection, which section has more roadside parking, etc. Human drivers can make a prediction based on experience and take evasive action or adopt a more reasonable driving strategy in advance. The drivable area detection method based on the network model generally trains the network model through a large amount of driving data in the training stage, and after the network model is fixed, the drivable area determination method will not be optimized gradually with the increase of the number of repeated driving. For repeated route driving that often occurs in regular driving, the network model cannot also refer to the historical driving experience in time.

[0088] The embodiment of the present application proposes an experience-based drivable area determination method, which stores the previous drivable path and continuously optimizes the historical drivable area information through an algorithm. When determining the drivable area during driving, the current drivable area and the historical drivable area information are combined and superimposed to obtain the optimized drivable area of this time.

[0089] In a possible implementation, the embodiment of the present application can store the historical drivable area information into the historical drivable area library in the form of a binary graph, so that the historical drivable area information can be stored persistently without occupying a large amount of space. The stored historical drivable area information can be continuously optimized with the number of repeated driving. Through the above-mentioned manner, the embodiment of the present application enables the drivable area determination to have a self-learning ability.

[0090] As Figure 3Fig. 1 shows a timing diagram for determining a drivable area according to an embodiment of the present application. T1, T2, T3, and the like represent the first, second, third, and the like time that a vehicle passes a same determination point. The intelligent vehicle passes the determination point for the first time at T1. The vehicle-mounted computing system performs initialization of the drivable area at T1 to obtain drivable area information at the determination point at T1. The drivable area is fused and updated at subsequent stages T2, T3, and the like. Specifically,

[0091] When the vehicle is driving, the drivable area is determined at the determination point passed by the vehicle. If no record information of the historical drivable area related to the determination point is matched in the historical drivable area library, the determination point is processed according to the operation at T1.

[0092] For T1, the detection module is first called to detect the drivable area and output the drivable area information. The drivable area information output at T1 is stored, including drivable area description information, GPS data, road feature points, and the like. The drivable area information output at T1 is historical drivable area information for initialization and is recorded in the historical area library. In a specific embodiment, for each recorded drivable area, the GPS data is stored as a reference and a unique label number can be assigned to identify each record for query.

[0093] When the vehicle passes the determination point for the second time (at T2), the historical drivable area information is read from the historical area library for operation at T2. Specifically, the GPS data of the determination point is matched with the GPS data in the historical library. After successful matching, the label number of the historical drivable area is output. The historical drivable area information corresponding to the label number is called out for fusion.

[0094] At T2, the detection module detects the current drivable area at the determination point. The current drivable area detected at T2 is fused with the historical drivable area by calling the fusion method. The drivable area is determined according to the drivable area determination method to obtain the current drivable area.

[0095] The current drivable area output is updated as the historical information. The updated content is the drivable area information corresponding to the label number.

[0096] When the vehicle passes the determination point again (at T3, T4, and the like), the operation at T2 is repeated to output the optimized drivable area information.

[0097] As shown in Fig. 1, the drivable area is determined at the determination point passed by the vehicle. If no record information of the historical drivable area related to the determination point is matched in the historical drivable area library, the determination point is processed according to the operation at T1. Figure 4As shown in a, it is a schematic diagram of the feasible region fusion provided by the embodiment of the present application. The feasible region recorded in the historical feasible region library when the vehicle is at the current determined location is indicated by a diagonal line. As shown in b, it is the current feasible region detected by the detection module, indicated by a reverse diagonal line. As shown in c, it is the optimized feasible region output after the historical feasible region is determined. The optimized feasible region can combine the historical experience information to preferentially output the minimum feasible region containing the optimal path. For example, the infeasible region of the right lane in a can be a ramp merging area, there can be a deceleration vehicle about to enter the ramp, and there is a high probability of an obstacle, while the middle lane is a straight lane, and the maximum probability is a smooth lane. For example, Figure 5 As shown in a, it is a schematic diagram of the feasible region fusion provided by the embodiment of the present application. The feasible region recorded in the historical feasible region library when the vehicle is at the current determined location is indicated by a diagonal line. As shown in b, it is the current feasible region detected by the detection module, indicated by a reverse diagonal line. As shown in c, it is the optimized feasible region output after the historical feasible region is determined. The optimized feasible region can combine the historical experience information to preferentially output the minimum feasible region containing the optimal path. For example, the infeasible region of the right lane in a can be a ramp merging area, there can be a deceleration vehicle about to enter the ramp, and there is a high probability of an obstacle, while the middle lane is a straight lane, and the maximum probability is a smooth lane. For example,

[0098] The embodiment of the present application provides a storage method of historical driving region information. The historical driving region information can adopt a low-resolution binary picture (for example, 128*128). Exemplarily, the granularity of the recorded historical feasible region information can be one meter, that is, one piece of data is stored every time the vehicle moves one meter, and the stored data includes three parts: feasible region information, GPS data and road feature points. The feasible region is stored as a binary picture, the road feature points are Scale-invariant feature transform (SIFT) point data after the camera removes moving objects according to target detection, and the road feature points can be stored as text; the GPS data is in a list, and a unique label number is used to associate the GPS data with the binary picture and the feature point data.

[0099] When the historical feasible region data is queried, the corresponding label number can be queried through the GPS data, and the corresponding binary picture and feature point data are obtained according to the label number. Exemplarily, the system can correspond the regions with a GPS data difference of not more than one meter to the same label number.

[0100] For each position (that is, each region distinguished according to the granularity), k pieces of historical feasible region information are stored, denoted as S n-1 , S n-2 , …, S n-k , respectively, representing the feasible region information output when passing through the region for the n-1, n-2, …, n-k time. Exemplarily, the value of k can be 10.

[0101] As shown in a, it is a schematic diagram of the feasible region fusion provided by the embodiment of the present application. The feasible region recorded in the historical feasible region library when the vehicle is at the current determined location is indicated by a diagonal line. As shown in b, it is the current feasible region detected by the detection module, indicated by a reverse diagonal line. As shown in c, it is the optimized feasible region output after the historical feasible region is determined. The optimized feasible region can combine the historical experience information to preferentially output the minimum feasible region containing the optimal path. For example, the infeasible region of the right lane in a can be a ramp merging area, there can be a deceleration vehicle about to enter the ramp, and there is a high probability of an obstacle, while the middle lane is a straight lane, and the maximum probability is a smooth lane. For example, Figure 6 As shown in a, it is a schematic diagram of the feasible region fusion provided by the embodiment of the present application. The feasible region recorded in the historical feasible region library when the vehicle is at the current determined location is indicated by a diagonal line. As shown in b, it is the current feasible region detected by the detection module, indicated by a reverse diagonal line. As shown in c, it is the optimized feasible region output after the historical feasible region is determined. The optimized feasible region can combine the historical experience information to preferentially output the minimum feasible region containing the optimal path. For example, the infeasible region of the right lane in a can be a ramp merging area, there can be a deceleration vehicle about to enter the ramp, and there is a high probability of an obstacle, while the middle lane is a straight lane, and the maximum probability is a smooth lane. For example,

[0102] Step 601: According to the position positioning information (GPS), query the historical drivable area library to obtain the information of the corresponding historical drivable area.

[0103] Step 602: Register the current drivable area with the historical drivable area, that is, align the historical drivable area with the current drivable area according to the feature points such as lane lines.

[0104] In a possible implementation, the embodiments of the present application can use the SIFT feature points to complete the registration operation by using the existing general feature extraction and registration method of computer vision. SIFT, namely, scale-invariant feature transform, is a description used in the field of image processing. The description has scale invariance and can detect key points in an image, and is a local feature description.

[0105] The SIFT feature detection mainly includes the following basic steps:

[0106] Scale space extreme detection: search for image positions in all scales, and identify potential interest points that are invariant to scale and rotation by using a Gaussian differential function.

[0107] Key point positioning: in each candidate position, a fine model is fitted to determine the position and scale. The selection of key points is based on their stability.

[0108] Direction determination: based on the gradient direction of the local image, one or more directions are assigned to each key point position. All subsequent operations on the image data are transformed relative to the direction, scale and position of the key points, thereby providing invariance to these transformations.

[0109] Key point description: in the neighborhood around each key point, the local image gradient is measured at the selected scale. These gradients are transformed into a representation that allows comparison of large local shape deformations and illumination changes. We store the road feature information, namely, the SIFT feature point information of the area;

[0110] Registration: In the registration stage, we also perform SIFT feature extraction on the current region; when the SIFT feature vectors of the two images are generated, the next step can use the Euclidean distance of the key point feature vectors as the similarity judgment measure of the key points in the two images. Take a key point of the current image, find the two closest key points in the historical image by traversal. In the two key points, if the nearest distance divided by the second nearest distance is less than a certain threshold, it is determined as a pair of matching points. According to the matching result of the above steps, select the 30 pairs of matching points with the highest matching value as the reference point pair, calculate the scale and direction parameters using the remaining matching point pairs, and then perform translation, rotation and scaling transformation according to the parameters, so as to obtain the historical drivable region after registration.

[0111] Step 603: superimpose the current drivable region and the historical drivable region.

[0112] Step 604: determine the drivable region of this time according to the superimposed drivable region.

[0113] The superimposed S com is output as the first priority. If there is no complete lane in S com , S is used as the output.

[0114] The lane line information can be obtained in two ways according to the difference of the application system. One way is to directly obtain it from the high-precision map according to the positioning result of the GPS, which is limited by the accuracy of the GPS positioning and whether the high-precision map is available. The other way is to directly extract it from the image of the current region by using the computer vision method, which can be further divided into the traditional image processing method (extracting edge information, etc.) and the deep learning (LaneNet, etc.) processing method. For the embodiments of the present application, the source of the lane line information is not limited and can come from the detection module or other existing ways in the system.

[0115] In order to determine whether there is a complete drivable lane, we also need the detection result of the detection module in the vehicle-mounted processing system. By comparing the detection result with the coordinates of the lane line information, we can determine whether there is a complete drivable lane in the drivable region. For example, the complete lane can be a lane within the sensing range of the vehicle as the reference, which has no obstacle target.

[0116] If S n-1 has no complete lane, S n is used as the output, that is, the output result of the drivable region detection module is used as the output.

[0117] The generation method of S com includes:

[0118] S n-1 , S n-2 , …, S n-k cumulative, according to the threshold (for example, k / 2) output binary matrix S:

[0119] S and the current drivable area S n and, get S com .

[0120] The generation method of S n-1 and the current drivable area S n and, get

[0121] S com , S n The following table is described:

[0122]

[0123] The embodiment also provides a method for updating historical drivable area information. If the current drivable area S n contains a complete lane, the historical drivable area information is updated, S n is kept as historical area information, S n-k previous historical area information (forget) is discarded, otherwise the historical drivable area is not updated (bad value is discarded).

[0124] As Figure 7 shown, a drivable area determination process schematic diagram provided by the embodiment of the application. It includes:

[0125] Step 701: The perception device obtains the environmental information around the vehicle.

[0126] Step 702: The detection module obtains the environmental information around the vehicle input by the perception device, and obtains lane line information.

[0127] Step 703: The detection module obtains positioning information, and obtains the current drivable area according to the environmental information.

[0128] Step 704: The query module combines the positioning information, and queries the historical drivable area information in the historical drivable area library.

[0129] Step 705: The fusion module combines the feature point matching, and registers the historical drivable area and the current drivable area (so that the lane lines are aligned).

[0130] Step 706: The current drivable area information is fused with the registered historical drivable area information, and S com area fusion is performed.

[0131] Step 707: In combination with the lane line detection result, if S com contains a complete lane, it is output as the determined drivable area.

[0132] Step 708: If S com does not contain a complete lane, region fusion is performed.

[0133] Step 709: In combination with the lane line detection result, if S contains a complete lane, it is output as the determined drivable area.

[0134] Step 710: If S does not contain a complete lane, S n is directly output as the determined historical drivable area.

[0135] Step 711: After the determined historical drivable area is output, the historical region is updated according to the method described in the foregoing embodiments of the present application, and the updated historical region is used as the historical drivable area library in subsequent determination.

[0136] The above process is repeated, and the drivable area determination is constantly self-updated and optimized with the increase in the number of repetitions.

[0137] The definition and storage method of the historical drivable area are used to represent and persist the historical experience information of the drivable area in autonomous driving. The foregoing fusion determination method is used to fuse the current drivable area and the historical drivable area, and output the drivable area where the optimal route is located. The update method of the historical drivable area is used to update the historical drivable area, and the forgetting rule and the discard rule can be set, so that the algorithm can forget distant information or discard old data after new drivable areas are added, thereby updating the road condition changes in time and avoiding the interference of individual abnormal values.

[0138] The embodiments of the present application provide a drivable area determination method and process based on historical information, and combine lane line detection to fuse the stored historical drivable area information and the current drivable area, and output the optimal drivable area.

[0139] The embodiments of the present application provide a storage method and update process of the historical drivable area, which converts the historical drivable area into a binary picture and stores it according to the current positioning information, thereby reducing the required storage space. The embodiments of the present application provide a self-learning update method of the historical drivable area, which stores the historical determined drivable area by encoding and combines it with the instant determination of the current drivable area, so that the selection of the drivable area can be gradually optimized with the increase in the historical record.

[0140] ​In specific embodiments, the storage manner of the historical drivable area information can be other manners, for example, storing a numerical matrix, and storing as map information, etc. The fusion determination manner of the historical drivable area and the current drivable area can be other manners, for example, adding the historical area and the current area, or weighted multiplication, etc. The updating method of the historical drivable area can also be other manners, for example, updating the historical drivable area by using the drivable area after the fusion determination.

[0141] The embodiments of the present application make the drivable area determination preferentially output the drivable area where the optimal route is located, improve the accuracy of the drivable area determination, reduce the exploration process of the automatic driving algorithm, improve the ride experience of the automatic driving, and meanwhile, the drivable area determination algorithm has self-learning ability, and realizes automatic optimization based on historical records.

[0142] Figure 8 A structural schematic diagram of a drivable area determination device provided by the embodiments of the present application, the drivable area determination device 800 can be a computing device, or a module in an intelligent driving system or a mobile data center MDC, and is used to realize the functions described in the foregoing process descriptions. The drivable area determination device 800 at least includes a processor 810, a communication interface 820 and a memory 830, which are connected with each other through a bus 840, wherein,

[0143] The specific implementation of the processor 810 performing various operations can refer to the specific operations of acquiring environment information and drivable area fusion in the foregoing method embodiments. The processor 810 can have various specific implementation forms, and the processor 810 performs relevant operations according to the program unit stored in the memory, and the program unit can be an instruction or a computer program. The processor 810 can be a central processing unit (CPU) or a graphics processing unit (GPU), and the processor 810 can also be a single-core processor or a multi-core processor.

[0144] The processor 810 can be a combination of a CPU and a hardware chip. The hardware chip can be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD can be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0145] The processor 810 can also be implemented by a logic device with built-in processing logic alone, such as an FPGA or a digital signal processor (DSP), etc.

[0146] The communication interface 820 can be a wired interface or a wireless interface, used for communication with other modules or devices. The wired interface can be an Ethernet interface, a controller area network (CAN) interface, a local interconnect network (LIN), and a FlexRay interface. The wireless interface can be a cellular network interface or a wireless local area network interface, etc. For example, the communication interface 820 in the embodiment of the present application can be specifically used to receive the environmental data collected by the perception device 153.

[0147] The bus 840 can be a CAN bus or other internal bus for interconnecting various systems or devices within the vehicle. The bus 840 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the figure, but it does not mean that there is only one bus or only one type of bus.

[0148] Optionally, the drivable area determination device can further include a memory 830, the storage medium of the memory 830 can be a volatile memory and a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM) used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced SDRAM (ESDRAM), synchlink DRAM (SLDRAM) and direct rambus RAM (DR RAM). The memory 830 can also be used to store program codes and data to facilitate the processor 810 to call the program codes stored in the memory 830 to realize the functions of the intelligent driving system in the foregoing embodiments. In addition, the drivable area determination device 800 can contain more or less components, or have different component configurations. Figure 8 More or less components are shown, or different component configurations are possible.

[0149] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0150] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. containing one or more available medium sets. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid state drive (SSD).

[0151] The steps in the method of the embodiments of the present application can be adjusted in sequence, combined or deleted according to actual needs; the modules in the device of the embodiments of the present application can be divided, combined or deleted according to actual needs.

[0152] The embodiments of the present application are described in detail above, and the principles and implementation modes of the present application are described by applying specific examples; the above embodiment descriptions are only used to help understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed; in view of the above, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A method for determining a drivable area for intelligent driving, characterized in that: include: The intelligent driving system obtains environmental information around the vehicle and determines the current drivable area; The intelligent driving system queries a historical drivable area database according to the location of the vehicle to obtain information on the corresponding historical drivable area; The intelligent driving system superimposes the current drivable area with the historical drivable area to obtain the current drivable area, and the drivable area is used for path planning assistance.

2. The method according to claim 1, wherein The intelligent driving system superimposes the current drivable area with the historical drivable area to obtain an optimized drivable area including: The intelligent driving system determines whether the superimposed drivable area includes a drivable lane that meets the first length. If so, the drivable lane that meets the first length is used as the drivable area this time.

3. The method according to claim 2, wherein When the superimposed drivable area does not include a drivable lane that meets the first length, the current drivable area is used as the current drivable area.

4. The method according to claim 2 or 3, wherein: When the drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas contains a drivable lane that meets the first length, the superimposed drivable area is used as the drivable area for this time; When the drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas does not include a drivable lane that meets the first length, determine whether the drivable area obtained by superimposing the current drivable area with the previous historical drivable area includes a drivable lane that meets the first length. If so, use the drivable area obtained by superimposing the current drivable area with the previous historical drivable area as the current drivable area, where k is a positive integer greater than or equal to 2.

5. The method according to any one of claims 1 to 4, characterized in that: Also includes: When the current drivable area includes a drivable lane that meets the first length, the current drivable area is updated to the historical drivable area library.

6. The method according to any one of claims 2 to 4, characterized in that: The first length represents the range perceived by the intelligent driving system, and the drivable lanes that meet the first length represent lanes that are drivable areas within the range perceived by the intelligent driving system.

7. The method according to any one of claims 1 to 6, wherein: The information of historical drivable areas includes historical drivable area records, positioning data, and road feature points.

8. The method according to any one of claims 1 to 7, wherein: The intelligent driving system stores information of historical drivable areas with a second length as a granularity.

9. The method according to any one of claims 1 to 8, wherein: After querying the historical drivable area database and obtaining information of the corresponding historical drivable area, the method further includes: The intelligent driving system aligns the current drivable area and the historical drivable area based on the road feature points at the vehicle's location, so that the road feature points of the current drivable area coincide with the road feature points included in the queried information of the historical drivable area on the map.

10. The method according to any one of claims 1 to 9, characterized in that: The historical drivable area obtained by query is greater than or equal to the perception range of the intelligent driving system.

11. An intelligent driving system, characterized in that: include: The detection module is used to obtain the environmental information around the vehicle and determine the current drivable area; A query module, configured to query a historical drivable area database according to the location of the vehicle and obtain information on the corresponding historical drivable area; A fusion module, configured to superimpose the current drivable area with the historical drivable area to obtain the current drivable area; The drivable area is used to assist in path planning.

12. The system according to claim 11, wherein The fusion module is specifically configured to determine whether the superimposed drivable area includes a drivable lane that meets a first length. If so, the drivable lane that meets the first length is used as the drivable area for this time.

13. The system according to claim 12, wherein: The fusion module is specifically configured to use the current drivable area as the current drivable area when the superimposed drivable area does not include a drivable lane that meets the first length.

14. The system according to claim 12 or 13, wherein: The fusion module is specifically configured to use the superimposed drivable area as the current drivable area when a drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas contains a drivable lane that meets the first length; The fusion module is specifically configured to determine whether a drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas contains a drivable lane that meets the first length when the drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas does not contain a drivable lane that meets the first length. If so, the drivable area obtained by superimposing the current drivable area with the previous historical drivable area is used as the drivable area for this time, where k is a positive integer greater than or equal to 2.

15. The system according to any one of claims 12 to 14, wherein: The fusion module is specifically configured to update the current drivable area to the historical drivable area library when the current drivable area includes a drivable lane that meets the first length.

16. The system according to any one of claims 12 to 15, wherein: The information of historical drivable areas includes historical drivable area records, positioning data, and road feature points.

17. The system according to claim 16, wherein: The fusion module is specifically used to align the current drivable area and the historical drivable area based on the road feature points at the vehicle's location, so that the road feature points of the current drivable area coincide with the road feature points included in the queried information of the historical drivable area on the map.

18. A smart car, characterized in that: Including processors, memory and sensing devices, The memory is used to store a historical drivable area library; The sensing device is used to obtain environmental information around the vehicle; The processor is configured to execute instructions to implement the method according to any one of claims 1 to 10.

Images