Method for determining drivable area, intelligent driving system and intelligent car

Abstract

An embodiment of the present application discloses a method for determining a drivable area for intelligent driving. The method determines the current drivable area based on environmental information around the vehicle's location. Based on the vehicle's location, a historical drivable area library is queried to obtain information on corresponding historical drivable areas. The current drivable area is superimposed on the historical drivable areas to obtain the current drivable area.

Description

Technical Field

[0001] The present application relates to the field of smart cars, and in particular to a method for determining a drivable area for smart driving, an intelligent driving system, and a smart car. Background Art

[0002] With economic development, the number of cars on the road has increased rapidly, and automotive technology is increasingly converging with computer technology. In recent years, smart cars have become a new trend in vehicle development, with an increasing number of vehicles adopting driver assistance, automated driving, or intelligent network driving systems. These systems intelligently detect obstacles and perceive the surrounding environment through onboard sensing devices such as image acquisition devices and sensors. They utilize onboard computing platforms (e.g., mobile data centers (MDCs)) to determine the vehicle's driving path and control its driving status.

[0003] Autonomous driving technology has broad application prospects and significant research significance. By using sensing equipment to detect roads and obstacles and autonomously perform driving operations, autonomous driving can improve driving safety, reduce traffic accidents, and minimize human and economic losses. Furthermore, autonomous driving can also work in conjunction with intelligent transportation systems to more rationally allocate road resources and alleviate urban congestion. Currently, autonomous driving technology is still in the research and testing phase, and drivable area detection is an essential component of both advanced driver assistance systems and autonomous driving.

[0004] Drivable area detection methods determine drivable areas based on input from current sensing devices. Existing technologies aim to instantly determine road conditions and identify drivable areas through machine learning and other methods. However, autonomous driving based on current drivable area detection technology only plans and controls drivable paths based on the detected current drivable area, limiting the accuracy of path planning. Summary of the Invention

[0005] The embodiments of the present application provide a method for determining a drivable area for intelligent driving, an intelligent driving system, and an intelligent vehicle, which solve the problem that the existing drivable area determination is instant, has limited accuracy, and cannot be self-learned.

[0006] In a first aspect, an embodiment of the present application provides a method for determining a drivable area for intelligent driving, comprising:

[0007] The intelligent driving system obtains environmental information around the vehicle and determines the current drivable area;

[0008] The intelligent driving system queries a historical drivable area database according to the location of the vehicle to obtain information on the corresponding historical drivable area;

[0009] The intelligent driving system superimposes the current drivable area with the historical drivable area to obtain the current drivable area.

[0010] The embodiment of the present application persists the previous drivable area determination results to a historical drivable area library and applies them to subsequent drivable area determination processes, thereby enabling the intelligent driving system to select the current driving path from the current drivable area to output the current drivable area including the optimal path.

[0011] The intelligent driving system determines whether the superimposed drivable area includes a drivable lane that meets a first length. If so, the drivable lane that meets the first length is used as the current drivable area. If the superimposed drivable area does not include a drivable lane that meets the first length, the current drivable area is used as the current drivable area.

[0012] In an embodiment of the present application, the historical drivable area of the vehicle's location is used as a factor in determining the current drivable area. In one possible implementation, the superimposed drivable area is output as the current drivable area only when a drivable lane that meets the first length exists in the superimposed drivable area. In this case, the drivable lane that meets the first length can be used as the optimal path, thereby making the output of the current drivable area more accurate.

[0013] In a possible implementation, the embodiment of the present application superimposes the current drivable area and the historical drivable area, and the current drivable area can be obtained as follows:

[0014] When the drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas contains a drivable lane that meets the first length, the superimposed drivable area is used as the drivable area for this time;

[0015] When the drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas does not include a drivable lane that meets the first length, determining whether the drivable area obtained by superimposing the current drivable area with the previous historical drivable area includes a drivable lane that meets the first length; if so, using the drivable area obtained by superimposing the current drivable area with the previous historical drivable area as the current drivable area;

[0016] Here, k is a positive integer greater than or equal to 2.

[0017] Of the two methods of superimposing the above-mentioned drivable areas, the preferred method is to superimpose the current drivable area with the previous k historical drivable areas. If there is a drivable lane that meets the first length in the superposition of the current drivable area and the previous k historical drivable areas, it means that the combination of the previous k historical drivable areas contains the optimal path, and the path is also likely to be the preferred path during the previous k driving processes; suboptimally, when there is no drivable lane that meets the first length after the current drivable area is superimposed with the previous k historical drivable areas, the previous drivable area can be superimposed with the previous historical drivable area. If there is a drivable lane that meets the first length in the superimposed area, it means that the drivable lane that meets the first length is a drivable lane in both the current and previous driving processes.

[0018] In a possible implementation, when the current drivable area includes a drivable lane that meets the first length, the current drivable area is updated to the historical drivable area library.

[0019] In another possible implementation, the embodiment of the present application can use the superimposed drivable areas to update the historical drivable area library. For example:

[0020] When the drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas contains a drivable lane that meets the first length, the superimposed drivable area is output as the current drivable area and recorded in the historical drivable area library;

[0021] When the drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas does not include a drivable lane that meets the first length, and the drivable area obtained by superimposing the current drivable area with the previous historical drivable area includes a drivable lane that meets the first length, the drivable area obtained by superimposing the current drivable area with the previous historical drivable area is output as the current drivable area and recorded in the historical drivable area library;

[0022] When none of the drivable areas obtained by the aforementioned superposition include a drivable lane that meets the first length, and the current drivable area includes a drivable lane that meets the first length, the current drivable area is output as the drivable area for this time and recorded in the historical drivable area library.

[0023] The intelligent driving system may also delete the record with the longest storage time when the number of records corresponding to a certain positioning data in the historical drivable area library exceeds k.

[0024] The above-mentioned updating method improves the timeliness of the data recorded in the historical drivable area library, and avoids storing too old data in the historical drivable area library, which affects the subsequent determination results of the drivable area.

[0025] In one example, the aforementioned first length represents the range perceived by the intelligent driving system, and the drivable lanes that meet the first length represent lanes that are drivable areas within the range perceived by the intelligent driving system.

[0026] In one possible implementation, a historical drivable area library records information about historical drivable areas, including historical drivable area records, positioning data, and road feature points. The intelligent driving system stores the historical drivable area information at a second length granularity. The second length may be an empirical value, such as 1 meter or 10 meters. A smaller second length requires more information about historical drivable areas to be stored, which occupies more storage space.

[0027] In one example, the historical drivable area information also includes a tag number. Each historical drivable area record corresponds to a tag number, and the tag number corresponds to positioning data. The intelligent driving system can query the positioning data for the tag number based on the vehicle's location data and output information about all historical drivable areas that match the tag number.

[0028] It is understood that the output historical drivable area is greater than or equal to the perception range of the intelligent driving system, that is, the output historical drivable area covers the perception range of the vehicle's forward direction. In this case, when the second length is less than the perception range radius of the intelligent driving system, the intelligent driving system obtains the historical drivable area of the vehicle's location that is greater than or equal to the perception range along the vehicle's forward direction, that is, obtains information on the historical drivable areas corresponding to multiple tag numbers in the forward direction, so that the range represented by the set of historical drivable areas corresponding to the multiple tag numbers is greater than or equal to the perception range of the vehicle's forward direction.

[0029] In a possible implementation, after querying the historical drivable area library and obtaining the corresponding historical drivable area information, the method further includes: the intelligent driving system aligning the current drivable area and the historical drivable area based on the road feature points at the vehicle's location, so that the road feature points of the current drivable area overlap with the road feature points included in the historical drivable area information obtained by query on the map.

[0030] In a second aspect, an embodiment of the present application provides an intelligent driving system, including:

[0031] The detection module is used to obtain the environmental information around the vehicle and determine the current drivable area;

[0032] A query module, configured to query a historical drivable area database according to the location of the vehicle and obtain information on the corresponding historical drivable area;

[0033] The fusion module is used to superimpose the current drivable area with the historical drivable area to obtain the current drivable area.

[0034] In a possible implementation, the fusion module is specifically configured to determine whether the superimposed drivable area includes a drivable lane that meets a first length; if so, the drivable lane that meets the first length is used as the drivable area for this time.

[0035] The fusion module is specifically configured to use the current drivable area as the current drivable area when the superimposed drivable area does not include a drivable lane that meets the first length.

[0036] The fusion module is specifically configured to use the superimposed drivable area as the current drivable area when a drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas contains a drivable lane that meets the first length;

[0037] The fusion module is specifically configured to determine whether a drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas contains a drivable lane that meets the first length when the drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas does not contain a drivable lane that meets the first length. If so, the drivable area obtained by superimposing the current drivable area with the previous historical drivable area is used as the drivable area for this time, where k is a positive integer greater than or equal to 2.

[0038] The fusion module is specifically configured to update the current drivable area to the historical drivable area library when the current drivable area includes a drivable lane that meets the first length.

[0039] The information of historical drivable areas includes historical drivable area records, positioning data, and road feature points.

[0040] The fusion module is specifically used to align the current drivable area and the historical drivable area based on the road feature points at the vehicle's location, so that the road feature points of the current drivable area coincide with the road feature points included in the queried information of the historical drivable area on the map.

[0041] In a third aspect, an embodiment of the present application further provides a smart car, including a processor, a memory, and a sensing device.

[0042] The memory is used to store a historical drivable area library;

[0043] The sensing device is used to obtain environmental information around the vehicle;

[0044] The processor is used to execute instructions to implement the method described in any specific implementation of the first aspect.

[0045] In a fourth aspect, an embodiment of the present application provides a drivable area determination device, comprising a processor, a communication interface, and a memory; the memory is used to store instructions, the processor is used to execute the instructions, and the communication interface is used to receive or send data; wherein, the processor executes the instructions to implement the method described in any specific implementation of the first aspect above.

[0046] In a fifth aspect, an embodiment of the present application provides a non-volatile computer storage medium, wherein the computer medium stores a computer program, and when the computer program is executed by a processor, the method described in the first aspect or any specific implementation of the first aspect is implemented.

[0047] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods.

[0048] The embodiment of the present application discloses a method for determining a drivable area for intelligent driving, which determines the current drivable area based on the environmental information around the vehicle's location, queries the historical drivable area library based on the vehicle's location, obtains the corresponding historical drivable area information, and superimposes the current drivable area with the historical drivable area to obtain the current drivable area. The embodiment of the present application provides a method for determining the current drivable area based on the information of the historical drivable area, and outputs the optimal drivable area by fusing the stored historical drivable area information with the current drivable area. Furthermore, the embodiment of the present application can be combined with road feature point detection to improve the accuracy of fusion. The embodiment of the present application can also update the historical drivable area and update the road condition changes to the historical records in a timely manner. Furthermore, the embodiment of the present application can also set a historical information aging mechanism, which discards at least one old data after updating the information of the current drivable area to the historical drivable area library, so that the algorithm can eliminate the influence of outdated information while avoiding interference from individual abnormal values. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a schematic diagram of the hardware structure of an in-vehicle computing system provided in an embodiment of the present application;

[0050] Figure 2 This is a schematic diagram of the logical structure of the intelligent driving system provided in an embodiment of the present application;

[0051] Figure 3 This is a schematic diagram of a timing flow of a drivable area fusion determination provided by an embodiment of the present application;

[0052] Figure 4This is a schematic diagram of the fusion of the current drivable area and the historical drivable area provided in an embodiment of the present application;

[0053] Figure 5 This is a schematic diagram of the perception range of an intelligent driving system provided in an embodiment of the present application.

[0054] Figure 6 This is a flow chart of a method for determining a drivable area for intelligent driving provided in an embodiment of the application;

[0055] Figure 7 This is a flow chart of another method for determining a drivable area for intelligent driving provided in an embodiment of the application;

[0056] Figure 8 This is a schematic diagram of a drivable area determination device provided in an embodiment of the application. DETAILED DESCRIPTION

[0057] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0058] The embodiments of the present application are applied to the field of smart cars, such as Figure 1 FIG2 is a schematic diagram of the hardware structure of an on-vehicle computing system applicable to an embodiment of the present application. The on-vehicle computing system may include an on-vehicle processing system 101 and devices / components / networks directly or indirectly connected to the on-vehicle processing system 101.

[0059] See also Figure 1 The on-board processing system 101 includes a processor 103 and a system memory 135. The processor 103 is connected to other components / interfaces on the on-board processing system 101 through a system bus 105. The processor 103 can be one or more processors, each of which can include one or more processor cores. The on-board processing system 101 is connected to other components on the vehicle through various interfaces (for example, an adapter 107, an I / O interface 115, a USB interface 125, etc.), such as a display 109, an interactive device 117, a multimedia device 121, a positioning device 123, and a perception device 153 (a camera and various sensors, etc.). The on-board processing system 101 is connected to an external network 127 through a network interface 129, and exchanges information with a server 149 through the external network 127. The network interface 129 can send and / or receive communication signals.

[0060] In the vehicle processing system 101, the system bus 105 is coupled to an input / output (I / O) bus 113 via a bus bridge 111. The I / O interface 115 communicates with various I / O devices, such as an interactive device 117 (e.g., a keyboard, mouse, touch screen, etc.), a multimedia device 121 (e.g., a compact disc read-only memory (CD-ROM), a multimedia interface, etc.), a positioning device 123, a universal serial bus (USB) interface 125, and a sensing device 153 (a camera capable of capturing still and moving digital video images).

[0061] The interactive device 117 is used to implement message interaction between the smart car and the driver. The driver can select the driving mode and driving style model of the smart car through the interactive device 117. In one possible implementation, the interactive device 117 can be integrated with the display 109.

[0062] Among them, the processor 103 can be any traditional processor, including a reduced instruction set computer (RISC) processor, a complex instruction set computer (CISC) processor or a combination of the above. Optionally, the processor can be a dedicated device such as an application specific integrated circuit (ASIC). Optionally, the processor 103 can be a neural network processor or a combination of a neural network processor and the above traditional processors. Optionally, the processor 103 may include: a main controller (also known as a central control) and an advanced driver assistance system controller. Among them, the main controller is the control center of the computer system. The advanced driver assistance system controller is used to control the route of autonomous driving or assisted autonomous driving, etc.

[0063] The display 109 may be any one or more display devices installed in the vehicle. For example, the display 109 may include a head-up display (HUD), an instrument panel, and a display specifically for passengers.

[0064] The vehicle processing system 101 can communicate with the server (deploying server) 149 via a network interface 129. The network interface 129 can be a hardware network interface, such as a network card. The network 127 can be an external network, such as the Internet, or an internal network, such as Ethernet or a virtual private network (VPN). Alternatively, the network 127 can be a wireless network, such as a WiFi network or a cellular network.

[0065] The memory interface 131 is coupled to the system bus 105. The memory interface is connected to the memory.

[0066] System memory 135 is coupled to system bus 105 . Data running in system memory 135 may include operating system 137 and application programs 143 .

[0067] Operating system 137 includes a shell 139 and a kernel 141. Shell 139 serves as an interface between the user and the operating system's kernel. The shell is the outermost layer of the operating system. It manages the interaction between the user and the operating system, waiting for user input, interpreting user input to the operating system, and processing various operating system outputs.

[0068] The kernel 141 consists of the parts of the operating system that manage memory, files, peripherals, and system resources. It interacts directly with the hardware and typically runs processes and provides inter-process communication, CPU time slice management, interrupts, memory management, and I / O management.

[0069] The intelligent driving system 143 includes programs related to controlling the automatic driving of the vehicle. For example, a program for processing the information about the vehicle's surrounding environment obtained by the on-board equipment, such as a program for implementing the drivable path determination method provided in the embodiment of the present application. Another example is a program for controlling the route or speed of an autonomous driving vehicle, a program for controlling the interaction between an autonomous driving vehicle and other autonomous driving vehicles on the road, etc. The intelligent driving system 143 may also exist on the system of the server 149. In one embodiment, when the intelligent driving system 143 needs to be executed, the on-board processing system 101 can download the installation program of the intelligent driving system 143 from the server 149.

[0070] The sensing device 153 is associated with the vehicle-mounted processing system 101. The sensing device 153 is used to detect the environment around the vehicle. For example, the sensing device 153 can detect animals, other vehicles, obstacles, crosswalks, lane lines, etc. around the vehicle. Further, the sensing device 153 can also detect the environment around the above-mentioned animals, vehicles, obstacles, crosswalks, etc., such as weather conditions, and the brightness of the surrounding environment. The sensing device may include a camera, an infrared sensor, a chemical detector, and a microphone. The sensing device 153 may also include a speed sensor for measuring the speed of the vehicle (i.e. Figure 1 The sensor 153 may also include a laser radar sensor for detecting reflected signals from laser signals transmitted by the laser radar, thereby generating a laser point cloud. The laser radar can be mounted above the vehicle to transmit laser signals.

[0071] The positioning device 123 includes a global positioning system (GPS), an inertial navigation system (INS), or other devices or subsystems for determining the vehicle's position.

[0072] In some embodiments of the present application, the processor 103 executes various instructions to implement various functions of the intelligent driving system 143. Specifically, Figure 2 As shown, an embodiment of the present application provides a structural example of the intelligent driving system 143 . The intelligent driving system 143 may include a detection module 1431 , a query module 1432 , a fusion module 1433 , and a vehicle control module 1434 .

[0073] Detection module 1431 is used to obtain environmental information surrounding the vehicle's location and determine the current drivable area. Specifically, detection module 1431 uses positioning device 123 and sensing device 153 to determine the surrounding environmental information, including information about obstacles in the area surrounding the vehicle (such as their location and size, including but not limited to the location, size, posture, and speed of physical objects such as people, vehicles, and roadblocks), as well as lane information. Detection module 1431 determines the current drivable area based on the surrounding environmental information.

[0074] The query module 1432 is used to query the historical drivable area library based on the vehicle's location to obtain information about the corresponding historical drivable area. The historical drivable area library records the drivable paths recorded when the vehicle passed the vehicle's location in history.

[0075] The fusion module 1433 is configured to superimpose the current drivable area with the historical drivable area to obtain the current drivable area.

[0076] The fusion module 1433 is specifically configured to determine whether the superimposed drivable area includes a drivable lane that meets the first length. If so, the drivable lane that meets the first length is used as the drivable area for this time.

[0077] Furthermore, the fusion module 1433 notifies the vehicle control module 1434 to control the vehicle's driving path according to the output current drivable area.

[0078] Memory 133 is used to store historical drivable area information and maps (such as high-precision maps) and update the stored content. Detection module 1431 retrieves map information for a specific area around the vehicle from memory 133 based on the vehicle's current location. This map information includes road markings within the specific area around the vehicle, such as road lines, lane markings, and stop lines. Detection module 1433 interacts with sensing device 153 to obtain information about obstacles and road markings in the area surrounding the vehicle and perform registration operations.

[0079] The memory 133 may be a memory specifically used to store maps, or may be a general-purpose memory, which is not limited in the present embodiment.

[0080] Each of the above modules (detection module 1431, query module 1432, fusion module 1433, and vehicle control module 1434) can be implemented by software and / or hardware. Moreover, any one or more of the modules can be set independently or integrated together. The embodiment of the present application does not specifically limit this. In one example, any one or more of these modules can be used as logical function modules in the main controller or in the ADAS controller.

[0081] The onboard computing system can be entirely located on the vehicle, or some of the processing logic can be located in a cloud network connected to the vehicle via a network. For example, the onboard processing system 101 can be located remotely from the autonomous vehicle and can communicate wirelessly with the autonomous vehicle. In other aspects, some of the processes described herein are executed on a processor located within the autonomous vehicle, while others are executed by a remote processor. For example, all or part of the functionality of the query module 1432 can be implemented on a server deployed in the cloud.

[0082] It should be noted that Figure 1 The computer system shown is only an example and does not limit the computer systems to which the embodiments of the present application are applicable.

[0083] The drivable area in the embodiments of the present application may include structured roads, semi-structured roads, unstructured roads, and other road areas that can be used for smart car driving. Structured roads generally have road edge lines and a single road structure, such as urban main roads, highways, national roads, provincial roads, etc. The structural layer of this road surface follows certain standards, and the color and material of the surface layer are uniform. Semi-structured roads refer to general non-standardized roads, where the surface layer has large differences in color and material, such as parking lots, squares, etc., and some branch roads. Unstructured roads have no structural layer and are natural road scenes. Autonomous driving needs to implement path planning, so it is necessary to implement the detection of drivable areas. In computer vision-based autonomous driving systems, drivable areas are generally detected based on image segmentation technology. For urban vehicles, autonomous driving can solve the detection and identification of structured and unstructured roads. For unmanned vehicles in the wild, it is necessary to solve the detection of unstructured roads.

[0084] There are many different detection methods for different environments. The basic method is to obtain basic structural features of the road surface based on road color, road model, and road texture characteristics. These features can further obtain potential information such as road edge lines and basic road directions (straight ahead, left turn, right turn, sharp left turn, sharp right turn). These features can be extracted through traditional segmentation extraction methods or machine learning methods.

[0085] The detection of drivable areas is mainly used to provide path planning assistance for autonomous driving. The detection can be for the entire road surface, or only partial road information can be extracted, such as the road direction or road midpoint or lane line within a certain area ahead. As long as the high-precision map can be combined to achieve road path planning and obstacle avoidance, it is not necessary to extract the complete drivable area of the road surface. For semi-structured or unstructured roads without clear road markings, there is no lane division on the road surface. The intelligent driving system 143 can further perform an analysis function to divide the road surface within the detection range into lanes and apply the drivable area detection results after lane division in the subsequent processing process.

[0086] For repetitive roads in autonomous driving scenarios, especially commuter routes, existing technologies cannot predict the road conditions of historical areas after multiple passes, like human drivers can. For example, which road section has potholes in some lanes, which road section has complex intersections, which road section has a lot of roadside parking, etc. Human drivers can make predictions based on experience and avoid them in advance or adopt more reasonable driving strategies. However, the drivable area detection method based on network models is generally trained in the training phase through a large amount of driving data. After the network model is fixed, the method for determining the drivable area will not be gradually optimized as the number of driving repetitions increases. For repetitive routes that often occur in routine driving, the network model cannot timely refer to previous historical driving experience.

[0087] This embodiment of the application proposes an experience-based method for determining drivable areas. This method stores previous drivable routes and continuously optimizes historical drivable area information through an algorithm. When determining a drivable area during driving, the current drivable area is combined and superimposed with the historical drivable area information to obtain the optimized drivable area for this time.

[0088] In one possible implementation, the present application embodiment can use a binary graph to store historical drivable area information in a historical drivable area library, allowing for persistent storage of historical drivable area information without taking up a large amount of space. The stored historical drivable area information can be continuously optimized with the number of repeated trips. Through this approach, the present application embodiment enables drivable area determination to have self-learning capabilities.

[0089] like Figure 3 The figure shows a timing diagram of a drivable area determination provided by an embodiment of the present application. T1, T2, T3, ... respectively represent the first, second, third ... time a vehicle passes through the same determination location. When a smart car passes through the determination location for the first time at T1, the onboard computing system performs drivable area initialization at the T1 stage, obtains drivable area information when passing through the determination point at the T1 stage, and performs fusion determination and update of the drivable area when passing through the determination point at the T1 stage. Specifically:

[0090] When the vehicle is driving, the drivable area is determined at the determination point it passes. If there is no matching record of the historical drivable area related to the location to be determined in the historical drivable area library, the operation of the T1 stage is performed at the determination point.

[0091] During the T1 phase, the detection module is first called to detect the drivable area and output the drivable area information. The drivable area information output during the T1 phase is stored, including drivable area description information, GPS data, and road feature points. The drivable area information output during the T1 phase serves as the initialized historical drivable area information and is recorded in a historical area library. In one specific embodiment, each recorded drivable area is stored based on GPS data and can be assigned a unique tag number to identify each record for easy querying.

[0092] When the vehicle passes the determination location for the second time (at stage T2), the historical drivable area information is read from the historical area library and the T2 operation is performed. Specifically, the GPS data of the determination location is matched with the GPS data in the historical library. If a match is successful, the tag number of the historical drivable area is output, and the historical drivable area information in the corresponding tag number is retrieved for integration.

[0093] In the T2 stage, the detection module detects the current drivable area at the determination location. For the current drivable area output by the T2 stage detection, the fusion method is called to fuse the current drivable area with the historical drivable area, and the current drivable area is obtained according to the drivable area determination method.

[0094] The drivable area output this time is used as historical information, and the historical area information is updated. The updated content is the drivable area information in the corresponding tag number.

[0095] When the vehicle passes the determination location again (T3, T4, ..., etc.), the operation of stage T2 is repeated to output the optimized drivable area information.

[0096] like Figure 4 As shown, it is a schematic diagram of a drivable area fusion provided by an embodiment of the present application. Figure a identifies the drivable area recorded in the historical drivable area library when the vehicle is at the current determination location, which is represented by a slash. Figure b is the current drivable area detected by the detection module, which is represented by a reverse slash. Figure c is the optimized drivable area output after combining the historical drivable area determination. The optimized drivable area can combine historical experience information and give priority to outputting the minimum drivable area containing the optimal path. For example, the non-drivable area of the right lane in Figure a may be the ramp merging area, where there may be a decelerating vehicle about to enter the ramp, and there is a high probability that there are obstacles, while the middle lane is a straight lane, and the highest probability is a clear lane. As Figure 5 The figure shows a schematic diagram of the range of perception of a perception device provided in an embodiment of the present application. The embodiment of the present application plans a drivable path within the above-mentioned perception range.

[0097] An embodiment of the present application provides a method for storing historical driving area information, and the historical driving area information can adopt a low-resolution binary image (for example: 128*128). Exemplarily, the granularity of the recorded historical drivable area information can be one meter, that is, a piece of data is stored every time the vehicle moves one meter, and the stored data includes three parts: drivable area information, GPS data, and road feature points. Among them, the drivable area is stored as a binary image, and the road feature points are the scale-invariant feature transform (SIFT) point data after the camera removes moving objects according to target detection. The road feature points can be stored as text; the GPS data exists in a list, and a unique tag number associates the GPS data with the binary image and feature point data.

[0098] When querying historical drivable area data, the system can use GPS data to find the corresponding tag number and obtain the corresponding binary image and feature point data based on the tag number. For example, the system can map areas with GPS data that differ by no more than one meter to the same tag number.

[0099] For each location (i.e., each area distinguished by granularity), store k copies of historical drivable area information, denoted as S n-1 , S n-2 ,…,S n-k , respectively represent the drivable area information output when the vehicle passes through the area for the n-1th, n-2th, ..., nkth times. For example, the value of k can be 10.

[0100] like Figure 6 As shown, an embodiment of the present application provides a flow chart of a method for fusion determination of a drivable area, the method comprising:

[0101] Step 601: According to the position positioning information (GPS), query the historical drivable area library to obtain the corresponding historical drivable area information.

[0102] Step 602: aligning the current drivable area with the historical drivable area, that is, aligning the historical drivable area with the current drivable area based on feature points such as lane lines.

[0103] In one possible implementation, the present application embodiment can use SIFT feature points to perform registration using existing computer vision feature extraction and registration methods. SIFT, or Scale-invariant Feature Transform (SIFT), is a description used in image processing. This description is scale-invariant and can detect key points in an image, serving as a local feature description.

[0104] SIFT feature detection mainly includes the following basic steps:

[0105] Scale-space extrema detection: Search image locations at all scales and identify potential scale- and rotation-invariant points of interest using Gaussian derivatives.

[0106] Keypoint localization: At each candidate location, a fine-tuned model is fitted to determine the location and scale. Keypoints are selected based on their stability.

[0107] Direction determination: Based on the local gradient direction of the image, one or more directions are assigned to each keypoint position. All subsequent operations on the image data are transformed relative to the direction, scale and position of the keypoint, thus providing invariance to these transformations.

[0108] Keypoint description: In the neighborhood around each keypoint, the local gradient of the image is measured at a selected scale. These gradients are transformed into a representation that allows for large local shape deformations and lighting changes. We store road feature information, namely the SIFT feature point information of the region;

[0109] Registration: During the registration phase, we also perform SIFT feature extraction on the current area; after the SIFT feature vectors of the two images are generated, the next step is to use the Euclidean distance of the key point feature vectors as a similarity determination metric for the key points in the two images. Take a key point in the current image and find the two closest key points in the historical image through traversal. Among these two key points, if the closest distance divided by the next closest distance is less than a certain threshold, they are determined to be a pair of matching points. Based on the matching results of the above steps, select the 30 pairs of matching points with the highest matching values as the reference point pairs, use the remaining matching point pairs to calculate the scale and direction parameters, and then perform translation, rotation, and scaling transformations based on the parameters to obtain the historical drivable area after registration.

[0110] Step 603: Overlay the current drivable area with the historical drivable area.

[0111] Step 604: Determine the current drivable area based on the superimposed drivable areas.

[0112] Superposition to obtain S com , as the highest priority output. If S com If there is no complete lane in , use as output.

[0113] There are two ways to obtain lane line information, depending on the differences in the application system. First, it can be obtained directly from the high-precision map based on the GPS positioning results. This method is limited by the accuracy of GPS positioning and whether a high-precision map is available. Second, it can be directly extracted from the image of the current area using computer vision methods. Computer vision methods can be divided into traditional image processing methods (extracting edge information, etc.) and deep learning (LaneNet, etc.). For the embodiments of the present application, the source of lane line information is not limited and can come from the detection module in the system or other existing methods.

[0114] To determine whether a complete drivable lane exists, we also need the detection results of the detection module in the vehicle processing system. By comparing the detection results with the coordinates of the lane line information, we can determine whether there is a complete drivable lane in the drivable area. For example, a complete lane can be a lane within the perception range of the vehicle with no obstacles.

[0115] If S n-1 If there is no complete lane, use S n As output, the output result of the drivable area detection module can be used as output.

[0116] S com The generation methods include:

[0117] S n-1 , S n-2 ,…,S n-k Accumulate and output the binary matrix S according to the threshold (for example, k / 2);

[0118] Combine S with the current drivable area S n Take and, we get S com .

[0119] The generation method is: S n-1 With the current drivable area S n Take and, get

[0120] S com , S n The description is as described in the following table:

[0121]

[0122] This embodiment also provides a method for updating the historical drivable area information. n If the complete lane is included in the , the historical drivable area information is updated and S is retained. nAs historical area information, discard S n-k The previous historical area information is forgotten, otherwise the historical drivable area is not updated (bad values are discarded).

[0123] like Figure 7 FIG. 1 is a schematic diagram of a drivable area determination process according to an embodiment of the present application. The process includes:

[0124] Step 701: The sensing device obtains environmental information around the vehicle.

[0125] Step 702: The detection module obtains the environmental information around the vehicle input by the sensing device and obtains the lane line information.

[0126] Step 703: The detection module obtains positioning information and obtains the current drivable area based on the environmental information.

[0127] Step 704: The query module searches for historical drivable area information in a historical drivable area library in combination with the positioning information.

[0128] Step 705: The fusion module combines feature point matching to align the historical drivable area with the current drivable area (so that lane lines, etc. are aligned).

[0129] Step 706: S-match the current drivable area information with the registered historical drivable area information. com Regional integration.

[0130] Step 707: Combine the lane line detection results, if S com If the complete lane is included in , it is output as the determined drivable area.

[0131] Step 708: If S com If the complete lane is not included, Regional integration.

[0132] Step 709: Combine the lane line detection results, if If the complete lane is included in , it is output as the determined drivable area.

[0133] Step 710: If If the complete lane is not included, then S n Output as the determined historical drivable area.

[0134] Step 711: After the determined historical drivable area is output, the historical area is updated according to the method described above in the embodiment of the present application, and the updated historical area is used as the historical drivable area library in subsequent determinations.

[0135] The above process is repeated, and the drivable area determination is continuously updated and optimized as the number of repetitions increases.

[0136] The definition and storage method of historical drivable areas is used to represent and persist historical experience information about drivable areas in autonomous driving. The aforementioned fusion judgment method is used to fuse the current drivable area with the historical drivable area and output the drivable area where the optimal route is located. The update method of historical drivable areas is used to update historical drivable areas. Forgetting and discarding rules can be set to allow the algorithm to forget long-standing information or discard old data after adding a new drivable area, thereby timely updating road condition changes and avoiding interference from individual outliers.

[0137] The embodiments of the present application provide a method and process for determining a drivable area based on historical information, and combine it with lane line detection to output the optimal drivable area by fusing the stored historical drivable area information with the current drivable area.

[0138] This embodiment of the application provides a storage method and update process for historical drivable areas. This method converts historical drivable areas into binary images and stores them according to current positioning information, reducing the storage space required. This embodiment of the application provides a self-learning update method for historical drivable areas. By encoding and storing historically determined drivable areas and combining them with the real-time determination of the current drivable area, the selection of drivable areas can be gradually optimized as the historical records increase.

[0139] In specific implementations, the historical drivable area information can be stored in other ways, such as as a numerical matrix or as map information. The historical drivable area can be integrated with the current drivable area in other ways, such as by adding the historical area to the current area or performing a weighted multiplication. The historical drivable area can also be updated in other ways, such as by using the fused and determined drivable area to update the historical drivable area.

[0140] The embodiment of the present application enables the determination of the drivable area to prioritize the output of the drivable area where the optimal route is located, thereby improving the accuracy of the drivable area determination, reducing the exploration process of the autonomous driving algorithm, and improving the autonomous driving riding experience. At the same time, the drivable area determination algorithm has self-learning capabilities, realizing automatic optimization based on historical records.

[0141] Figure 8This is a schematic diagram of the structure of a drivable area determination device provided in an embodiment of the present application. The drivable area determination device 800 can be a computing device or a module in an intelligent driving system or a mobile data center MDC, used to implement the functions described in the above processes. The drivable area determination device 800 includes at least: a processor 810, a communication interface 820 and a memory 830. The processor 810, the communication interface 820 and the memory 830 are interconnected via a bus 840, wherein:

[0142] The specific implementation of the various operations performed by processor 810 can refer to the specific operations such as obtaining environmental information and integrating drivable areas in the above-mentioned method embodiments. Processor 810 can have various specific implementation forms. Processor 810 performs related operations based on program units stored in memory. Program units can be instructions, or computer programs. Processor 810 can be a central processing unit (CPU) or a graphics processing unit (GPU). Processor 810 can also be a single-core processor or a multi-core processor.

[0143] Processor 810 may be a combination of a CPU and a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0144] The processor 810 may also be implemented using a logic device with built-in processing logic, such as an FPGA or a digital signal processor (DSP).

[0145] The communication interface 820 can be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface can be an Ethernet interface, a controller area network (CAN) interface, a local interconnect network (LIN), or a FlexRay interface. The wireless interface can be a cellular network interface or a wireless local area network interface. For example, in the embodiment of the present application, the communication interface 820 can be specifically used to receive environmental data collected by the sensing device 153.

[0146] The bus 840 may be a CAN bus or other internal bus that interconnects various systems or devices in the vehicle. The bus 840 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0147] Optionally, the drivable area determination device may further include a memory 830. The storage medium of the memory 830 may be a volatile memory and a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM). The memory 530 can also be used to store program code and data, so that the processor 810 calls the program code stored in the memory 830 to implement the functions of the intelligent driving system in the aforementioned embodiments. In addition, the drivable area determination device 800 may include a plurality of memory modules compared to the drivable area determination device 800. Figure 8 Show more or fewer components, or configure components differently.

[0148] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0149] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any other combination. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium. The semiconductor medium can be a solid state drive (SSD).

[0150] The steps in the method of the embodiment of the present application can be adjusted in order, combined or deleted according to actual needs; the modules in the device of the embodiment of the present application can be divided, combined or deleted according to actual needs.

[0151] The above is a detailed introduction to the embodiments of the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method and core idea of the present application. At the same time, for those skilled in the art, according to the idea of the present application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for determining a drivable area for intelligent driving, characterized in that: include: The intelligent driving system obtains environmental information around the vehicle and determines the current drivable area; The intelligent driving system queries a historical drivable area database according to the location of the vehicle to obtain information on the corresponding historical drivable area; The intelligent driving system superimposes the current drivable area with the historical drivable area to obtain a current drivable area. The current drivable area is used to determine the drivable path of the vehicle. The current drivable area is the overlapping part of the current drivable area and the historical drivable area.

2. The method according to claim 1, wherein The intelligent driving system superimposes the current drivable area with the historical drivable area to obtain an optimized drivable area including: The intelligent driving system determines whether the superimposed drivable area includes a drivable lane that meets the first length. If so, the drivable lane that meets the first length is used as the drivable area this time.

3. The method according to claim 2, wherein When the superimposed drivable area does not include a drivable lane that meets the first length, the current drivable area is used as the current drivable area.

4. The method according to claim 2 or 3, wherein: When the drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas contains a drivable lane that meets the first length, the superimposed drivable area is used as the drivable area for this time; When the drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas does not include a drivable lane that meets the first length, determine whether the drivable area obtained by superimposing the current drivable area with the previous historical drivable area includes a drivable lane that meets the first length. If so, use the drivable area obtained by superimposing the current drivable area with the previous historical drivable area as the current drivable area, where k is a positive integer greater than or equal to 2.

5. The method according to any one of claims 1 to 3, characterized in that: Also includes: When the current drivable area includes a drivable lane that meets the first length, the current drivable area is updated to the historical drivable area library.

6. The method according to claim 2 or 3, wherein: The first length represents the range perceived by the intelligent driving system, and the drivable lanes that meet the first length represent lanes that are drivable areas within the range perceived by the intelligent driving system.

7. The method according to claim 1, wherein The information of historical drivable areas includes historical drivable area records, positioning data, and road feature points.

8. The method according to claim 1 or 7, wherein: The intelligent driving system stores information of historical drivable areas with a second length as a granularity.

9. The method according to any one of claims 1 to 3, characterized in that: After querying the historical drivable area database and obtaining information of the corresponding historical drivable area, the method further includes: The intelligent driving system aligns the current drivable area and the historical drivable area based on the road feature points at the vehicle's location, so that the road feature points of the current drivable area coincide with the road feature points included in the queried information of the historical drivable area on the map.

10. The method according to any one of claims 1 to 3, characterized in that: The historical drivable area obtained by query is greater than or equal to the perception range of the intelligent driving system.

11. An intelligent driving system, characterized in that: include: The detection module is used to obtain the environmental information around the vehicle and determine the current drivable area; A query module, configured to query a historical drivable area database according to the location of the vehicle and obtain information on the corresponding historical drivable area; A fusion module is used to superimpose the current drivable area with the historical drivable area to obtain a current drivable area, wherein the current drivable area is used to determine the drivable path of the vehicle, and the current drivable area is the overlapping part of the current drivable area and the historical drivable area.

12. The system according to claim 11, wherein The fusion module is specifically configured to determine whether the superimposed drivable area includes a drivable lane that meets a first length. If so, the drivable lane that meets the first length is used as the drivable area for this time.

13. The system according to claim 12, wherein: The fusion module is specifically configured to use the current drivable area as the current drivable area when the superimposed drivable area does not include a drivable lane that meets the first length.

14. The system according to claim 12 or 13, wherein: The fusion module is specifically configured to use the superimposed drivable area as the current drivable area when a drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas contains a drivable lane that meets the first length; The fusion module is specifically used to determine whether a drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas contains a drivable lane that meets the first length when the drivable area obtained by superimposing the current drivable area with the previous k historical drivable areas does not contain a drivable lane that meets the first length. If so, the drivable area obtained by superimposing the current drivable area with the previous historical drivable area is used as the drivable area for this time, where k is a positive integer greater than or equal to 2.

15. The system according to any one of claims 11 to 13, wherein: The fusion module is specifically configured to update the current drivable area to the historical drivable area library when the current drivable area includes a drivable lane that meets the first length.

16. The system according to claim 11, wherein The information of historical drivable areas includes historical drivable area records, positioning data, and road feature points.

17. The system according to claim 16, wherein: The fusion module is specifically used to align the current drivable area and the historical drivable area based on the road feature points at the vehicle's location, so that the road feature points of the current drivable area coincide with the road feature points included in the queried information of the historical drivable area on the map.

18. A smart car, characterized in that: Including processors, memory and sensing devices, The memory is used to store a historical drivable area library; The sensing device is used to obtain environmental information around the vehicle; The processor is configured to execute instructions to implement the method according to any one of claims 1 to 10.

Images