A method for constructing a map and a computing device

Abstract

The embodiment of the present application discloses a method for constructing a map and a computing device. The constructed map can be applied to the laser processing field in the field of autonomous driving, and specifically can be applied to intelligent entities for intelligent driving (such as smart cars, smart connected cars, and autonomous driving cars). The method includes: performing two operations on each frame of laser point cloud obtained, one is target feature extraction, constructing a function that fits each target feature, and obtaining restrictions corresponding to each function, these functions and the restrictions corresponding to the functions constitute a main map, and the other is constructing a sub-map according to each frame of laser point cloud (obtained by splicing the occupied grid sub-maps corresponding to each frame of laser point cloud). The constructed sub-map is the occupied grid map (OGM), and then by establishing an index of OGM on the main map, a composite laser map of the main map and OGM is constructed. The composite laser map has low storage capacity requirements and retains more feature information for matching and positioning.

Description

Technical Field

[0001] The present application relates to the field of laser processing, and in particular to a method and computing device for constructing a map. Background Art

[0002] As one of the key technologies of autonomous driving, positioning technology achieves accurate positioning of autonomous vehicles by integrating various positioning methods with multiple sensor data, so that autonomous vehicles can obtain their exact positions. Accurate positioning is an indispensable function for autonomous vehicles. Among them, laser sensors such as lidar and 3D laser scanners have high measurement accuracy, so the laser point clouds obtained by these laser sensors are widely used in accurate positioning.

[0003] The premise of achieving precise positioning through laser sensors is to first obtain a map built based on laser point clouds (which can be referred to as laser maps), and then match the laser point clouds obtained by the laser sensor in real time with the laser map to achieve positioning. At present, in the implemented solutions, there are two main ways to build laser maps: one is to directly build the original laser point cloud into a laser map, and directly match the laser point cloud obtained by the laser sensor in real time with the laser map during subsequent positioning; the other is to compress the three-dimensional laser point cloud into two-dimensional information, and construct a two-dimensional occupancy grid map (OGM) based on the compressed two-dimensional information. The two-dimensional OGM constitutes the laser map.

[0004] However, both of the above methods have defects. The laser map obtained by the first method requires too large a storage capacity and is difficult to reuse in engineering. The second method compresses the three-dimensional laser point cloud into two-dimensional information, which loses many features and affects the subsequent positioning accuracy. Summary of the invention

[0005] The embodiments of the present application provide a method and computing device for constructing a map, by establishing an index of an occupied grid map on a main map, and constructing a composite laser map of the main map and the occupied grid map, thereby reducing the storage capacity of the composite laser map and retaining more feature information for subsequent matching and positioning.

[0006] Based on this, the embodiments of the present application provide the following technical solutions:

[0007] In the first aspect, the embodiment of the present application provides a method for constructing a map, which can be applied to the field of laser processing in the field of autonomous driving, for example, it can be applied to intelligent bodies (such as smart cars, smart connected cars) for intelligent driving, the method comprising: the computing device first acquires the laser point cloud data (also referred to as laser point cloud) required for constructing the map, and extracts features from the acquired laser point cloud to obtain target features, the target features being laser points that meet preset conditions extracted from the laser point cloud data, the laser points including the coordinates of the laser points and the reflection intensity of the laser points. For example, a laser sensor in a standard posture may acquire a frame of laser point cloud at different geographical locations, and after obtaining a total of n frames of laser point cloud, the computing device may extract features from the n frames of laser point cloud to obtain target features; or a laser sensor in a standard posture may acquire a frame of laser point cloud at different geographical locations, and send it to the computing device for feature extraction until all n frames of laser point cloud are processed, and the specific method for the computing device to process the laser point cloud is not limited here. After the corresponding target features are extracted from the laser point cloud, a function fitting each target feature can be constructed and the restriction conditions of the function can be obtained. For example, assuming that three target features are extracted from the first frame of the laser point cloud, two of which are line features and one is a surface feature, then functions fitting these three target features can be constructed separately (a total of three functions), and the restriction conditions of each function can also be obtained (a total of three restriction conditions). Similarly, the above processing can be performed on each frame of the laser point cloud to obtain the functions and restriction conditions corresponding to the target features of all n frames of laser point clouds. These functions and restriction conditions constitute the main map. For example, assuming that there are 100 frames of laser point cloud, and 800 target features are extracted from them, then 800 functions and constraints corresponding to the 800 functions can be obtained. These 800 target features and 800 constraints constitute the main map. It should be noted here that the 800 target features extracted are target features that have been screened and merged. For example, 10 target features and 8 target features are extracted from 2 different frames of laser point cloud, respectively. Some of the target features may represent the same thing (such as the same street lamp, roadblock, etc.). In this case, it is necessary to merge the same target features into one target feature. Assuming that 2 of the 10 target features extracted from the previous frame of laser point cloud are the same as 2 of the 8 target features extracted from the next frame of laser point cloud, then only 2 target features in one of them need to be retained. The target features extracted subsequently are all processed in this way, which will not be repeated here. The computing device will also construct a sub-map for each frame of the acquired laser point cloud, and the constructed sub-map is an occupied grid sub-map (OGM).After performing the above processing on each frame of laser point cloud, a main map and an OGM are obtained. At this time, it is necessary to link the main map and OGM to form a composite laser map. Specifically, the computing device can establish an index of OGM on the main map to combine and obtain a composite laser map.

[0008] In the above-mentioned embodiment of the present application, the computing device performs two operations on the acquired laser point cloud. One operation is target feature extraction, constructing a function that fits each target feature, and obtaining the restriction conditions corresponding to each function. These functions and the restriction conditions corresponding to the functions constitute the main map. The other operation is to construct a sub-map OGM based on the laser point cloud. After that, by establishing an index of OGM on the main map, a composite laser map of the main map and OGM is constructed, which reduces the storage capacity of the composite laser map while retaining more feature information for subsequent matching and positioning.

[0009] In a possible design of the first aspect, the computing device constructs the OGM based on the laser point cloud. The computing device first constructs a corresponding occupation grid sub-map for each frame of the laser point cloud obtained, or constructs a corresponding occupation grid sub-map for several consecutive frames of laser point clouds. For example, assuming that there are 100 frames of laser point clouds, 100 occupation grid sub-maps can be constructed (one-to-one or many-to-one, without limitation, only for illustration). The process of constructing each occupation grid sub-map is as follows: after setting the length and width of the occupation grid sub-map (i.e., setting the size of the occupation grid sub-map) and the grid resolution, the computing device projects each frame of the laser point cloud obtained in the laser coordinate system into the corresponding occupation grid sub-map. If there is no laser point in a grid, it is considered to be empty. If there is at least one laser point, it is considered that there is an obstacle corresponding to the grid. Therefore, for a grid, the probability that it is empty is expressed as p(s=1), and the probability that there is an obstacle is expressed as p(s=0), and the sum of the probabilities of the two is 1. After that, the computing device performs a series of mathematical transformations on each frame of laser point cloud projected to the occupied grid submap, and locates the grid as occupied or idle according to the probability of whether each grid is occupied, wherein the center position of the occupied grid submap is the origin O of the occupied grid submap. Similarly, the above processing is performed for each frame of laser point cloud, so that all n frames of laser point cloud correspond to an occupied grid submap (a total of n), and then the occupied grid submaps corresponding to these n frames of laser point cloud are spliced ​​to obtain a complete OGM.

[0010] In the above-mentioned implementation of the present application, it is specifically explained how to construct the corresponding occupancy grid sub-map from the laser point cloud, and how to splice these occupancy grid sub-maps into a complete OGM, which is feasible.

[0011] In a possible design of the first aspect, the computing device may establish an index of the OGM on the main map as follows: first, the computing device converts the center position (i.e., the origin) of the occupied grid sub-map corresponding to each laser point cloud into a coordinate value in the universal transverse mercator grid system (UTM) coordinate system; and then, adds each origin coordinate value on the main map as an index label of the occupied grid sub-map corresponding to each frame of the laser point cloud.

[0012] In the above implementation of the present application, a specific implementation method of how to establish a connection between the main map and the OGM is explained, that is, adding index tags of each grid sub-map to the main map. This implementation method is easy to implement and simple to operate.

[0013] In a possible design of the first aspect, the computing device may obtain a restriction condition for the function by: obtaining a value interval of an independent variable corresponding to the function; or obtaining a value of a target independent variable in the function, wherein the target independent variable includes the coordinates of a target laser point, and the target laser point belongs to the above-mentioned target feature.

[0014] In the above-mentioned implementation manner of the present application, specific expressions of the restriction conditions of several functions are given, which are flexible and optional.

[0015] In a possible design of the first aspect, the OGM may include: the height of obstacles in a first grid and the average value of the reflection intensity of laser points falling into the first grid, and the first grid is any occupied grid in the occupied grid map. That is to say, each occupied grid (i.e., the first grid) in the OGM obtained in the embodiment of the present application stores the average height of the laser points falling into the grid (i.e., the average height of the obstacles in the grid) and the average reflection intensity (i.e., the average value of the reflection intensity of the laser points falling into the grid).

[0016] In the above-mentioned embodiment of the present application, the difference between the provided OGM and the existing OGM is that it not only stores the average height of the obstacles corresponding to the occupied grid, but also stores the average reflection intensity of the laser points falling into the occupied grid. The existing OGM only stores the average height of the obstacles, and the reflection intensity of the laser points is stored elsewhere. The advantage of such storage in the embodiment of the present application is that it is easy to search and more convenient in actual application.

[0017] In a possible design of the first aspect, the computing device may store the height of the occupied grid obstacle in the OGM as integer data, and may store the average reflection intensity of the laser points falling in the occupied grid in the OGM as integer data, and may store both the height of the occupied grid obstacle in the OGM and the average reflection intensity of the laser points falling in the occupied grid as integer data, wherein the integer data is numerical data that does not contain a decimal part, is only used to represent integers, and is stored in binary form.

[0018] In the above implementation mode of the present application, it is explained that the data occupying the grid storage in the OGM can be integer data. The existing solutions store data in the form of floating-point data. Integer data occupies less storage space than floating-point data (theoretically, the storage capacity occupied by integer data is 1 / 4 of that of floating-point data). Therefore, the advantage of doing so in the implementation mode of the present application is that storage capacity can be saved.

[0019] In a possible design of the first aspect, each occupied grid in the OGM obtained in the embodiment of the present application can be divided into partially occupied and fully occupied. Full occupation refers to obstacles extending from the ground to a certain height (such as general buildings such as office buildings and residential buildings), and partial occupation refers to buildings such as bridge tunnels, tunnels, viaducts, and aerial crosswalks that occupy a part of the space, which can also be called suspended obstacles. For these two types of occupation, when an occupied grid in the OGM is fully occupied, that is, the obstacle of the grid is a general obstacle, then the obstacle height stored in the grid is the height of the upper edge of the obstacle from the ground; when an occupied grid in the OGM is partially occupied, that is, the obstacle of the grid is a suspended obstacle, then the obstacle height stored in the grid is the first height of the lower edge of the obstacle from the ground and the second height of the upper edge of the obstacle from the ground.

[0020] In the above-mentioned implementation of the present application, the difference between the improved OGM provided by the embodiment of the present application and the existing OGM is that the obstacles in the occupied grid are classified into general buildings and suspended obstacles, and different heights are stored for different types of obstacles. At the same time, the average reflection intensity of the laser points falling into the occupied grid is also stored, while the existing OGM is considered to be completely occupied if a grid is occupied. The advantage of storing the height of obstacles in this way in the embodiment of the present application is that more detailed features of the obstacles are retained, thereby improving the accuracy of subsequent positioning.

[0021] In a possible design of the first aspect, the target feature of the embodiment of the present application is essentially to extract some special laser points from a frame of laser point cloud. The extracted target features are generally line features and surface features, wherein the line feature is used to indicate that the laser points extracted from the laser point cloud data are located on the same straight line, and the surface feature is used to indicate that the laser points extracted from the laser point cloud data are located on the same plane.

[0022] In the above-mentioned embodiments of the present application, some conditions that the extracted target features meet are specifically described, and the extracted target features have key features that are beneficial to subsequent positioning.

[0023] A second aspect of the embodiments of the present application provides a computing device, which has the function of implementing the method of the first aspect or any possible implementation of the first aspect. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0024] A third aspect of an embodiment of the present application provides a computing device, which may include a memory, a processor, and a bus system, wherein the memory is used to store programs, and the processor is used to call the programs stored in the memory to execute the method of the first aspect of the embodiment of the present application or any possible implementation method of the first aspect.

[0025] A fourth aspect of the present application provides a computer-readable storage medium, which stores instructions. When the computer-readable storage medium is run on a computer, the computer can execute the method of the first aspect or any possible implementation of the first aspect.

[0026] A fifth aspect of an embodiment of the present application provides a computer program, which, when executed on a computer, enables the computer to execute the method of the first aspect or any possible implementation of the first aspect.

[0027] The sixth aspect of the embodiment of the present application provides a chip, which includes at least one processor and at least one interface circuit, the interface circuit is coupled to the processor, the at least one interface circuit is used to perform a transceiver function, and send instructions to the at least one processor, the at least one processor is used to run a computer program or instruction, which has the function of implementing the method of the first aspect or any possible implementation of the first aspect, the function can be implemented by hardware, can also be implemented by software, can also be implemented by a combination of hardware and software, the hardware or software includes one or more modules corresponding to the above functions. In addition, the interface circuit is used to communicate with other modules outside the chip, for example, the interface circuit can send the composite laser map obtained by the processor on the chip to various intelligent driving (such as unmanned driving, assisted driving, etc.) intelligent agents for motion planning (such as driving behavior decision, global path planning, etc.). BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 A schematic diagram of OGMs with different resolutions provided in an embodiment of the present application;

[0029] Figure 2 A schematic diagram of an OGM of a certain region constructed according to an embodiment of the present application;

[0030] Figure 3 A schematic diagram of the overall architecture of an autonomous driving vehicle provided in an embodiment of the present application;

[0031] Figure 4 A schematic diagram of the structure of an autonomous driving vehicle provided in an embodiment of the present application;

[0032] Figure 5 A flowchart of a method for constructing a map provided in an embodiment of the present application;

[0033] Figure 6 A schematic diagram of extracting target features and constructing functions based on a frame of laser point cloud provided in an embodiment of the present application;

[0034] Figure 7 A schematic diagram of nine occupancy grid sub-maps constructed corresponding to nine frames of laser point clouds provided in an embodiment of the present application being spliced ​​into one OGM;

[0035] Figure 8 A schematic diagram of the constructed OGM storage data type provided in an embodiment of the present application;

[0036] Fig. 9 Another schematic diagram of the constructed OGM storage data type provided in an embodiment of the present application;

[0037] Fig.10 A schematic diagram of a process for constructing a composite laser map provided in an embodiment of the present application;

[0038] Fig.11 A schematic diagram of the actual application process of the constructed composite laser map provided in the embodiment of the present application;

[0039] Fig.12 A schematic diagram of a computing device provided in an embodiment of the present application;

[0040] Fig.13 Another schematic diagram of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0041] The embodiments of the present application provide a method and computing device for constructing a map, by establishing an index of an occupied grid map on a main map, and constructing a composite laser map of the main map and the occupied grid map, thereby reducing the storage capacity of the composite laser map and retaining more feature information for subsequent matching and positioning.

[0042] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, which is only to describe the distinction mode adopted by the objects of the same attributes when describing in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0043] The embodiments of this application involve a lot of knowledge about laser point clouds, maps, etc. In order to better understand the solutions of the embodiments of this application, the following first introduces the relevant terms and concepts that may be involved in the embodiments of this application. It should be understood that the interpretation of relevant terms and concepts may be limited due to the specific circumstances of the embodiments of this application, but it does not mean that this application is limited to this specific situation. The specific circumstances in different embodiments may also be different, which is not specifically limited here.

[0044] (1) Laser point cloud

[0045] Laser point cloud can also be called laser point cloud data. The laser information received by laser sensors such as laser radar and 3D laser scanner is presented in the form of point cloud, and the point data set of the surface of the measured object obtained by the measuring instrument is called point cloud. If the measuring instrument is a laser sensor, then the obtained point cloud is called laser point cloud (generally, 32-line laser will have tens of thousands of laser points at the same time). The laser information contained in the laser point cloud can be recorded as [x, y, z, intensity]. The laser information represents the three-dimensional coordinates of the target position hit by each laser point in the laser coordinate system and the reflection intensity of the laser point.

[0046] (2) Universal Transverse Mercator Grid System (UTM) coordinate system

[0047] UTM coordinates are a type of plane rectangular coordinates. This coordinate grid system and the projection it is based on have been widely used in topographic maps, as a reference grid for satellite images and natural resource databases, and other applications that require precise positioning. For example, the precise positioning of autonomous vehicles generally uses UTM coordinates.

[0048] The UTM projection is an elliptical cylinder with the horizontal axis cutting the earth ellipsoid. The center line of the elliptical cylinder is located on the equatorial plane of the ellipsoid and passes through the ellipsoid mass. Thus, the points on the ellipsoid are projected onto the elliptical cylinder. The lengths of the two secant circles on the UTM projection map remain unchanged, that is, the two standard meridian circles. The middle of the two secant circles is the central meridian circle. The length of the central meridian after projection is 0.9996 times its length before projection. The scale factor k = length after projection / actual length before projection. The difference in longitude between the standard secant and the central meridian is 1.6206°, or 1°37′14.244″. UTM longitude zones range from 1 to 60, with 58 zones spanning 6° east-west. Longitude zones cover all areas of the Earth's mid-latitudes from 80°S to 84°N. There are 20 UTM latitude zones, each spanning 8° north-south, identified by letters C to X (without I and O). Zones A, B, Y, and Z are not part of the system; they cover the Antarctic and Arctic regions.

[0049] The format of UTM coordinates is: longitude zone latitude zone east north, where east represents the projected distance from the central meridian of the longitude zone, and north represents the projected distance from the equator. The units of these two values ​​are both meters. For example, the result of using UTM to represent the longitude / latitude coordinates (61.44, 25.40) is 35V 414668 6812844, while the result of longitude / latitude coordinates (-47.04, -73.48) is 18G 615471 4789269.

[0050] (3) Occupancy grid map (OGM)

[0051] OGM is a commonly used map representation method for robots. Robots often use laser sensors, and sensor data is noisy. For example, it is impossible to detect an accurate value when using a laser sensor to detect how far the obstacle in front is from the robot. For example, at one angle, if the accurate value is 4 meters, then the obstacle is detected at 3.9 meters at the current moment, but 4.1 meters at the next moment. The positions at both distances cannot be considered obstacles. To solve this problem, OGM is used, such as Figure 1It is a diagram of two OGMs with different resolutions. The black dots are laser points. All laser points mapped in the OGM constitute a laser point cloud. In practical applications, the OGM size generally used is 300*300, that is, it consists of 300*300 small grids (i.e. grids). The size of each grid (i.e. length*width, which means how many meters each grid corresponds to in the vehicle coordinate system) is called the resolution of the OGM. The higher the resolution, the smaller the grid size, then the fewer laser points in a laser point cloud acquired by the laser sensor at a certain moment that fall in a specific grid, such as Figure 1 As shown in the figure on the left, it falls on the gray bottom grid ( Figure 1 There are 4 laser points in the 6th row and 11th column of the left figure. On the contrary, the lower the resolution and the larger the grid size, the fewer laser points will fall in a specific grid at the same time. Figure 1 As shown in the figure on the right, it falls on the gray bottom grid ( Figure 1 There are 9 laser points in the 4th row and 7th column of the right figure. For general maps, a certain point on the map either has an obstacle or not, but in OGM, at a specific moment, if there is no laser point in a grid, it is considered empty, and if there is at least one laser point, it is considered that there is an obstacle corresponding to the grid. Therefore, for a grid, the probability of it being empty is expressed as p (s = 1), and the probability of having an obstacle is expressed as p (s = 0), and the sum of the two probabilities is 1. After that, the laser point cloud acquired at different times is mapped to the OGM, and after a series of mathematical transformations, the grid is positioned as occupied or idle according to the probability of whether each grid is occupied. It should be noted that, generally speaking, the center position of the OGM is the origin of the OGM, such as Figure 1 The triangle shown in the left figure indicates the origin of the OGM.

[0052] For ease of understanding, the following illustrates a two-dimensional OGM (the grid is generally at the decimeter level) constructed in an embodiment of the present application. Each occupied grid stores the average height of the laser points falling into the grid (i.e., the average height of the obstacles in the grid) and the average reflection intensity. Figure 2 , Figure 2 The diagram shows the OGM of a certain region constructed in the embodiment of the present application.

[0053] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0054] The map constructed based on the laser point cloud in the embodiment of the present application can be applied to the scene of motion planning (such as driving behavior decision, global path planning, etc.) of various intelligent driving (such as unmanned driving, assisted driving, etc.) intelligent agents. Taking the intelligent agent as an autonomous driving vehicle as an example, the overall architecture of the autonomous driving vehicle is first described. For details, please refer to Figure 3 , Figure 3 The diagram shows a top-down hierarchical system architecture. There may be defined interfaces between systems to transmit data between systems to ensure the real-time and integrity of the data. The following is a brief introduction to each system:

[0055] (1) Environmental Perception System

[0056] Environmental perception is the most basic part of intelligent driving vehicles. Whether it is making driving behavior decisions or global path planning, it needs to be based on environmental perception. According to the real-time perception results of the road traffic environment, corresponding judgments, decisions and plans are made to enable the vehicle to achieve intelligent driving.

[0057] The environmental perception system mainly uses various sensors to obtain relevant environmental information, so as to complete the construction of the environmental model and the knowledge expression of the traffic scene. The sensors used include cameras, single-line radar (SICK), four-line radar (IBEO), three-dimensional laser radar (HDL-64E), etc. Among them, the camera is mainly responsible for traffic light detection, lane line detection, road sign detection, vehicle recognition, etc.; the laser sensor is mainly responsible for the detection, recognition and tracking of dynamic / static obstacles and its own precise positioning. For example, the laser emitted by the three-dimensional laser radar generally collects external environmental information at a frequency of 10FPS and returns the laser point cloud at each moment. Finally, the real-time laser point cloud obtained is sent to the autonomous decision-making system for further decision-making and planning.

[0058] (2) Autonomous decision-making system

[0059] The autonomous decision-making system is a key component of intelligent driving vehicles. The system is mainly divided into two core subsystems: behavioral decision-making and motion planning. Among them, the behavioral decision-making subsystem mainly obtains the global optimal driving route by running the global planning layer to clarify the specific driving task, and then according to the current real-time road information (i.e. Figure 3 Specifically, in the embodiment of the present application, the autonomous decision-making system outputs the position, orientation and other information of the objects around the vehicle according to the real-time laser point cloud of each frame sent by the environmental perception system, and uses a matching algorithm to match the map constructed in advance in the embodiment of the present application (i.e. Figure 3The laser map in the image is matched to achieve accurate positioning of the vehicle. Common matching algorithms include direct point cloud matching (such as iterative closest point (ICP) algorithm), probability matching (such as normal distribution transform (NDT) algorithm), filter matching (such as histogram filtering), feature matching, etc.

[0060] Finally, based on road traffic rules and driving experience, reasonable driving behavior is decided according to the positioning of the vehicle, as well as the position and orientation of surrounding objects, and the driving behavior instructions are sent to the motion planning subsystem. The motion planning subsystem further plans a feasible driving trajectory based on indicators such as safety and stability according to the received driving behavior instructions and the current environmental perception information, and sends it to the control system.

[0061] (3) Control system

[0062] Specifically, the control system is also divided into two parts: a control subsystem and an execution subsystem. The control subsystem is used to convert the feasible driving trajectory generated by the autonomous decision-making system into specific execution instructions for each execution module and pass them to the execution subsystem; the execution subsystem receives the execution instructions from the control subsystem and sends them to each control object to reasonably control the vehicle's steering, braking, throttle, gear, etc., so that the vehicle can drive automatically to complete the corresponding driving operations.

[0063] It should be noted that Figure 3 The overall architecture of the autonomous driving vehicle shown is for illustration only. In actual applications, it may include more or fewer systems / subsystems or modules, and each system / subsystem or module may include multiple components, which are not specifically limited here.

[0064] In order to further understand this solution, based on Figure 3 Corresponding to the overall architecture of the autonomous driving vehicle, the embodiments of the present application will also be combined with Figure 4 For an introduction to the specific functions of the various structures inside an autonomous vehicle, please refer to Figure 4 , Figure 4A schematic diagram of the structure of an autonomous driving vehicle provided in an embodiment of the present application, wherein the autonomous driving vehicle 100 is configured in a fully or partially autonomous driving mode, for example, the autonomous driving vehicle 100 can control itself while in the autonomous driving mode, and can determine the current state of the vehicle and its surrounding environment through human operation, determine the possible behavior of at least one other vehicle in the surrounding environment, and determine the confidence level corresponding to the possibility of other vehicles performing possible behaviors, and control the autonomous driving vehicle 100 based on the determined information. When the autonomous driving vehicle 100 is in the autonomous driving mode, the autonomous driving vehicle 100 can also be set to operate without human interaction.

[0065] The autonomous vehicle 100 may include various subsystems, such as a travel system 102, a sensor system 104 (e.g., Figure 3 The camera, SICK, IBEO, laser radar, etc. in the autonomous driving system 104 are all modules in the sensor system 104), a control system 106, one or more peripheral devices 108, and a power supply 110, a computer system 112, and a user interface 116. Optionally, the autonomous driving vehicle 100 may include more or fewer subsystems, and each subsystem may include multiple components. In addition, each subsystem and component of the autonomous driving vehicle 100 may be interconnected by wire or wirelessly.

[0066] The travel system 102 may include components that provide powered movement for the autonomous vehicle 100. In one embodiment, the travel system 102 may include an engine 118, a power source 119, a transmission 120, and wheels / tires 121.

[0067] Among them, the engine 118 can be an internal combustion engine, an electric motor, an air compression engine, or a combination of other types of engines, for example, a hybrid engine consisting of a gasoline engine and an electric motor, and a hybrid engine consisting of an internal combustion engine and an air compression engine. The engine 118 converts the energy source 119 into mechanical energy. Examples of energy sources 119 include gasoline, diesel, other petroleum-based fuels, propane, other compressed gas-based fuels, ethanol, solar panels, batteries, and other sources of electricity. The energy source 119 can also provide energy for other systems of the autonomous driving vehicle 100. The transmission 120 can transmit the mechanical power from the engine 118 to the wheels 121. The transmission 120 may include a gearbox, a differential, and a drive shaft. In one embodiment, the transmission 120 may also include other devices, such as a clutch. Among them, the drive shaft may include one or more shafts that can be coupled to one or more wheels 121.

[0068] The sensor system 104 may include several sensors that sense information about the environment around the autonomous driving vehicle 100. For example, the sensor system 104 may include a positioning system 122 (the positioning system may be a global positioning GPS system, or a Beidou system or other positioning systems), an inertial measurement unit (IMU) 124, a radar 126, a laser rangefinder 128, and a camera 130. The sensor system 104 may also include sensors of the internal systems of the monitored autonomous driving vehicle 100 (for example, an in-vehicle air quality monitor, a fuel gauge, an oil temperature gauge, etc.). The sensing data from one or more of these sensors can be used to detect objects and their corresponding characteristics (position, shape, direction, speed, etc.). This detection and recognition is a key function for the safe operation of the autonomous autonomous driving vehicle 100. In the embodiment of the present application, the laser sensor is a very important perception module in the sensor system 104.

[0069] Among them, the positioning system 122 can be used to estimate the geographic location of the autonomous driving vehicle 100. In the embodiment of the present application, the laser sensor can be used as one of the positioning systems 122 to achieve accurate positioning of the autonomous driving vehicle 100, and the IMU 124 is used to sense the position and orientation changes of the autonomous driving vehicle 100 based on inertial acceleration. In one embodiment, the IMU 124 can be a combination of an accelerometer and a gyroscope. The radar 126 can use radio signals to sense objects in the surrounding environment of the autonomous driving vehicle 100, and can be specifically expressed as a millimeter wave radar or a laser radar. In some embodiments, in addition to sensing objects, the radar 126 can also be used to sense the speed and / or forward direction of the object. The laser rangefinder 128 can use lasers to sense objects in the environment where the autonomous driving vehicle 100 is located. In some embodiments, the laser rangefinder 128 may include one or more laser sources, a laser scanner, and one or more detectors, as well as other system components. The camera 130 can be used to capture multiple images of the surrounding environment of the autonomous driving vehicle 100. The camera 130 can be a static camera or a video camera.

[0070] The control system 106 controls the operation of the autonomous vehicle 100 and its components. The control system 106 may include various components, including a steering system 132 , a throttle 134 , a brake unit 136 , a computer vision system 140 , a lane control system 142 , and an obstacle avoidance system 144 .

[0071] Among them, the steering system 132 can be operated to adjust the forward direction of the autonomous driving vehicle 100. For example, in one embodiment, it can be a steering wheel system. The throttle 134 is used to control the operating speed of the engine 118 and thus control the speed of the autonomous driving vehicle 100. The brake unit 136 is used to control the deceleration of the autonomous driving vehicle 100. The brake unit 136 can use friction to slow down the wheel 121. In other embodiments, the brake unit 136 can convert the kinetic energy of the wheel 121 into electric current. The brake unit 136 can also take other forms to slow down the rotation speed of the wheel 121 to control the speed of the autonomous driving vehicle 100. The computer vision system 140 can be operated to process and analyze the images captured by the camera 130 in order to identify objects and / or features in the surrounding environment of the autonomous driving vehicle 100. The objects and / or features may include traffic signals, road boundaries, and obstacles. The computer vision system 140 can use object recognition algorithms, structure from motion (SFM) algorithms, video tracking, and other computer vision technologies. In some embodiments, the computer vision system 140 can be used to map the environment, track objects, estimate the speed of objects, and so on. The route control system 142 is used to determine the route and speed of the autonomous driving vehicle 100. In some embodiments, the route control system 142 may include a lateral planning module 1421 and a longitudinal planning module 1422, which are respectively used to determine the route and speed for the autonomous driving vehicle 100 in combination with data from the obstacle avoidance system 144, GPS 122, and one or more predetermined maps. The obstacle avoidance system 144 is used to identify, evaluate, avoid, or otherwise cross obstacles in the environment of the autonomous driving vehicle 100, and the aforementioned obstacles may specifically be actual obstacles and virtual moving bodies that may collide with the autonomous driving vehicle 100. In one example, the control system 106 may include components other than those shown and described in addition or in lieu thereof. Alternatively, some of the components shown above may be reduced.

[0072] The autonomous vehicle 100 interacts with external sensors, other vehicles, other computer systems, or users through the peripheral devices 108. The peripheral devices 108 may include a wireless communication system 146, an onboard computer 148, a microphone 150, and / or a speaker 152. In some embodiments, the peripheral devices 108 provide a means for the user of the autonomous vehicle 100 to interact with the user interface 116. For example, the onboard computer 148 may provide information to the user of the autonomous vehicle 100. The user interface 116 may also operate the onboard computer 148 to receive user input. The onboard computer 148 may be operated through a touch screen. In other cases, the peripheral devices 108 may provide a means for the autonomous vehicle 100 to communicate with other devices located in the vehicle. For example, the microphone 150 may receive audio (e.g., voice commands or other audio input) from the user of the autonomous vehicle 100. Similarly, the speaker 152 may output audio to the user of the autonomous vehicle 100. The wireless communication system 146 may communicate wirelessly with one or more devices directly or via a communication network. For example, the wireless communication system 146 may use 3G cellular communication, such as CDMA, EVDO, GSM / GPRS, or 4G cellular communication, such as LTE. Or 5G cellular communication. The wireless communication system 146 may communicate using a wireless local area network (WLAN). In some embodiments, the wireless communication system 146 may communicate directly with the device using an infrared link, Bluetooth, or ZigBee. Other wireless protocols, such as various vehicle communication systems, for example, the wireless communication system 146 may include one or more dedicated short range communications (DSRC) devices, which may include public and / or private data communications between vehicles and / or roadside stations.

[0073] The power source 110 can provide power to various components of the autonomous vehicle 100. In one embodiment, the power source 110 can be a rechargeable lithium-ion or lead-acid battery. One or more battery packs of such batteries can be configured as a power source to provide power to various components of the autonomous vehicle 100. In some embodiments, the power source 110 and the energy source 119 can be implemented together, such as in some all-electric vehicles.

[0074] Some or all of the functions of the autonomous vehicle 100 are controlled by a computer system 112. The computer system 112 may include at least one processor 113 that executes instructions 115 stored in a non-transitory computer-readable medium such as a memory 114. The computer system 112 may also be a plurality of computing devices that control individual components or subsystems of the autonomous vehicle 100 in a distributed manner. The processor 113 may be any conventional processor, such as a commercially available central processing unit (CPU). Alternatively, the processor 113 may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Although Figure 1 The processor, memory, and other components of the computer system 112 in the same block are functionally illustrated, but it will be appreciated by those skilled in the art that the processor, or memory, may actually include multiple processors, or memories that are not stored in the same physical housing. For example, the memory 114 may be a hard drive or other storage medium located in a housing different from the computer system 112. Therefore, references to the processor 113 or memory 114 will be understood to include references to a collection of processors or memories that may or may not operate in parallel. Different from using a single processor to perform the steps described herein, some components such as the steering assembly and the deceleration assembly may each have their own processor that performs only calculations related to the functions specific to the component.

[0075] In various aspects described herein, the processor 113 may be located remotely from the autonomous vehicle 100 and in wireless communication with the autonomous vehicle 100. In other aspects, some of the processes described herein are performed on a processor 113 disposed within the autonomous vehicle 100 while others are performed by the remote processor 113, including taking the necessary steps to perform a single maneuver.

[0076] In some embodiments, the memory 114 may contain instructions 115 (e.g., program logic) that can be executed by the processor 113 to perform various functions of the autonomous vehicle 100, including those described above. The memory 114 may also contain additional instructions, including instructions to send data to, receive data from, interact with, and / or control one or more of the travel system 102, the sensor system 104, the control system 106, and the peripheral devices 108. In addition to the instructions 115, the memory 114 may also store data such as road maps, route information, the vehicle's location, direction, speed, and other such vehicle data, as well as other information. This information can be used by the autonomous vehicle 100 and the computer system 112 during the operation of the autonomous vehicle 100 in autonomous, semi-autonomous, and / or manual modes. A user interface 116 is used to provide information to or receive information from a user of the autonomous vehicle 100. Optionally, the user interface 116 may include one or more input / output devices within the set of peripheral devices 108, such as a wireless communication system 146, an onboard computer 148, a microphone 150, and a speaker 152.

[0077] The computer system 112 may control functions of the autonomous vehicle 100 based on input received from various subsystems (e.g., the travel system 102, the sensor system 104, and the control system 106) and from the user interface 116. For example, the computer system 112 may utilize input from the control system 106 in order to control the steering system 132 to avoid obstacles detected by the sensor system 104 and the obstacle avoidance system 144. In some embodiments, the computer system 112 may be operable to provide control over many aspects of the autonomous vehicle 100 and its subsystems.

[0078] Optionally, one or more of the above components may be installed or associated separately from the autonomous vehicle 100. For example, the memory 114 may be partially or completely separate from the autonomous vehicle 100. The above components may be communicatively coupled together in a wired and / or wireless manner.

[0079] Optionally, the above components are only examples. In actual applications, the components in the above modules may be added or deleted according to actual needs. Figure 4 It should not be understood as a limitation of the embodiments of the present application. An autonomous vehicle traveling on a road, such as the autonomous vehicle 100 above, can identify objects in its surrounding environment to determine adjustments to the current speed. The objects can be other vehicles, traffic control devices, or other types of objects. In some examples, each identified object can be considered independently, and based on the respective characteristics of the object, such as its current speed, acceleration, spacing from the vehicle, etc., it can be used to determine the speed to be adjusted by the autonomous vehicle.

[0080] Optionally, the autonomous vehicle 100 or a computing device associated with the autonomous vehicle 100 may be Figure 4 The computer system 112, computer vision system 140, and memory 114 can predict the behavior of the identified objects based on the characteristics of the identified objects and the state of the surrounding environment (e.g., traffic, rain, ice on the road, etc.). Optionally, each identified object depends on the behavior of each other, so all identified objects can also be considered together to predict the behavior of a single identified object. The autonomous driving vehicle 100 can adjust its speed based on the predicted behavior of the identified objects. In other words, the autonomous driving vehicle 100 can determine what stable state the vehicle will need to adjust to (e.g., accelerate, decelerate, or stop) based on the predicted behavior of the objects. In this process, other factors can also be considered to determine the speed of the autonomous driving vehicle 100, such as the lateral position of the autonomous driving vehicle 100 in the road on which it is traveling, the curvature of the road, the proximity of static and dynamic objects, etc. In addition to providing instructions to adjust the speed of the autonomous vehicle, the computing device may also provide instructions to modify the steering angle of the autonomous vehicle 100 so that the autonomous vehicle 100 follows a given trajectory and / or maintains a safe lateral and longitudinal distance from objects near the autonomous vehicle 100 (e.g., cars in adjacent lanes on the road).

[0081] The above-mentioned autonomous driving vehicle 100 can be a car, a truck, a motorcycle, a bus, a ship, an airplane, a helicopter, a lawn mower, an entertainment vehicle, an amusement park vehicle, construction equipment, a tram, a golf cart, a train, and a cart, etc., and the embodiments of the present application are not particularly limited.

[0082] The embodiment of the present application provides a method for constructing a map, and the constructed map can be applied to various intelligent driving (such as unmanned driving, assisted driving, etc.) intelligent agents (such as, Figure 3 , Figure 4 For scenarios where motion planning (such as driving behavior decision-making, global path planning, etc.) is performed by the corresponding autonomous driving vehicle's overall architecture and various structural functional modules, please refer to Figure 5 , Figure 5 A flowchart of a method for constructing a map provided in an embodiment of the present application may include the following steps:

[0083] 501. Extract target features based on the laser point cloud data, where the target features are laser points that meet preset conditions and are extracted from the laser point cloud data.

[0084] The computing device will first obtain the laser point cloud required for building the map, and perform feature extraction on each frame of the laser point cloud obtained to obtain the target feature, which is the laser point that meets the preset conditions extracted from the laser point cloud data, and the laser point includes the coordinates of the laser point and the reflection intensity of the laser point. For example, a laser sensor in a standard posture may obtain a frame of laser point cloud at different geographical locations, and after obtaining n frames of laser point cloud, the computing device may perform feature extraction on each frame of the n frames of laser point cloud to obtain the target feature; or a laser sensor in a standard posture may obtain a frame of laser point cloud at different geographical locations and send it to the computing device for feature extraction until all n frames of laser point cloud are processed. The specific method of processing laser point cloud by the computing device is not limited here.

[0085] It should be noted that, in some embodiments of the present application, the target features described in the embodiments of the present application are essentially some special laser points extracted from a frame of laser point cloud, and the extracted target features are generally line features and surface features, among which the line features are used to indicate that the laser points extracted from the laser point cloud data are located on the same straight line, and the surface features are used to indicate that the laser points extracted from the laser point cloud data are located on the same plane.

[0086] In the embodiments of the present application, the method for extracting target features from laser point clouds is generally implemented by adopting a variety of screening methods, which can be specifically summarized as laser point cloud curvature feature extraction, that is, by calculating the curvature of the laser point cloud and filtering according to the curvature, it is determined in a frame of laser point cloud which laser points are located on the same plane and which laser points are located on the same straight line.

[0087] 502. Construct a function that fits the target feature and obtain constraints of the function, wherein the function and the constraints constitute a main map.

[0088] After extracting the corresponding target features from each frame of laser point cloud, a function fitting each target feature can be constructed and the restriction conditions of the function can be obtained. For example, assuming that 3 target features are extracted from the first frame of laser point cloud, 2 of which are line features and 1 is a surface feature, then the functions fitting these 3 target features can be constructed separately (3 functions in total), and the restriction conditions of each function can also be obtained (3 restriction conditions in total). Similarly, the above processing is performed for each frame of laser point cloud, and the functions and restriction conditions corresponding to the target features of all n frames of laser point cloud can be obtained. These functions and restriction conditions constitute the main map. For example, assuming that there are 100 frames of laser point cloud and 800 target features are extracted from the total, then 800 functions and restriction conditions corresponding to the 800 functions can be obtained. These 800 target features and 800 restriction conditions constitute the main map.

[0089] It should be noted here that the 800 target features extracted are target features that have been screened and merged. For example, 10 target features and 8 target features are extracted from two different frames of laser point cloud respectively. Some of the target features may represent the same object (such as the same street lamp, roadblock, etc.). In this case, it is necessary to merge these same target features into one target feature. Assuming that 2 of the 10 target features extracted from the previous frame of laser point cloud are the same as 2 of the 8 target features extracted from the next frame of laser point cloud, then only 2 target features in one of the copies need to be retained. The target features extracted subsequently are processed in this way and will not be elaborated here.

[0090] It should be noted that, in some embodiments of the present application, each time a frame of laser point cloud is obtained, the target features are extracted from the current frame of laser point cloud (assuming that 3 target features are extracted), and then a function that fits the target features is constructed and the corresponding constraints of the function are obtained (e.g., 3 functions and 3 constraints are constructed), and the above processing is performed on each frame of laser point cloud obtained until all laser point clouds are processed; in other embodiments of the present application, all n frames (e.g., 100 frames) of laser point clouds are obtained first, and then target features are extracted from all n frames of laser point clouds (assuming that 800 target features are extracted after screening), and then a function that fits all target features is constructed and the corresponding constraints of the function are obtained (e.g., 800 functions and 800 constraints are constructed).

[0091] To facilitate understanding of the above steps 501 and 502, the following example is given for illustration. Figure 6 , assuming Figure 6 The left picture is a frame of laser point cloud (visualization) corresponding to a certain geographic location obtained by the computing device. First, the computing device extracts the target features of the acquired frame of laser point cloud, that is, finds which laser points are on the same plane and which laser points are on the same straight line. After extracting these target features, the corresponding function is fitted according to these target features. Figure 6 Take the two extracted target features as an example (in fact, a frame of laser point cloud may extract more than a dozen target features, which is only for illustration). Assume that the two target features extracted by the computing device are one line feature and one surface feature, then the computing device performs fitting on the two extracted target features respectively. Assume that the fitting result of the line feature is as follows: Figure 6 The function f shown in the figure is fitted to the surface features as follows Figure 6If there are no restrictions on function f and function g, then function f represents an infinite straight line and function g represents a plane without boundaries. Therefore, it is necessary to obtain the restrictions of these two functions respectively. The restrictions are to make the obtained functions fit the extracted target features, such as Figure 6 The function f with restrictions and the function g with restrictions can just fit the extracted target features.

[0092] It should be noted that in some embodiments of the present application, the restriction condition of the function may be the value range of the function corresponding to the independent variable. Figure 6 Taking an example for illustration, assuming that the expression of the fitted function f is f(x)=ax+b, where x is the three-dimensional coordinate of the laser point, and a and b are parameter variables. According to the extracted target features, the values ​​of the parameter variables a and b can be determined (within the fitting error range). However, the function f with the parameter values ​​determined is a straight line in the three-dimensional space with no extension. Therefore, the minimum interval within which the values ​​of the three-dimensional coordinates of the extracted target features fall, that is, the value interval of the independent variable corresponding to the function f, can be used as the restriction condition of the function f.

[0093] It should also be noted that, in some embodiments of the present application, the constraint condition of the function may also be the coordinates of some key laser points, for example, Figure 6 The target feature (the target feature is fitted with function f) is located at both ends of Figure 6 The coordinates of some key laser points located at the plane corner points of the target feature (the target feature is fitted by the function g). The specific form of the restriction condition of the function is not limited here.

[0094] It should be noted that, in some implementations of the present application, the coordinate system of each function and constraint condition constituting the main map may adopt the universal transverse mercator griddystem (UTM) coordinate system.

[0095] It should also be noted that since the main map is composed of multiple functions and corresponding constraints, each function can be numbered for easy distinction, that is, assigned an ID for easy subsequent search. The numbering method is not limited here.

[0096] It should also be noted that in some embodiments of the present application, feature information for visual positioning can also be added to the main map, that is, the perception information obtained by various types of sensors (such as image information taken in real time by a camera installed on an autonomous driving vehicle) can be integrated on the main map to make subsequent positioning more accurate.

[0097] 503. Construct an occupancy grid map according to the laser point cloud data.

[0098] Each frame of laser point cloud obtained needs to perform two operations. One operation is to extract target features, construct a function that fits each target feature, and obtain the restriction conditions corresponding to each function, so as to obtain the main map (as described in steps 501 to 502). The other operation is to construct a sub-map, which is the occupied grid sub-map (OGM).

[0099] Here is a detailed introduction on how to construct OGM based on each frame of laser point cloud. For each frame of laser point cloud obtained, the computing device will first construct a corresponding occupation grid submap, or, for several frames of continuous laser point cloud, construct a corresponding occupation grid submap. For example, assuming there are 100 frames of laser point cloud, then 100 occupation grid submaps can be constructed (one-to-one or many-to-one, without limitation, only for illustration). The process of constructing each occupation grid submap is as follows: after setting the length and width of the occupation grid submap (i.e., setting the size of the occupation grid submap) and the grid resolution, the computing device projects each frame of laser point cloud obtained in the laser coordinate system into the corresponding occupation grid submap. If there is no laser point in a grid, it is considered empty, and if there is at least one laser point, it is considered that there is an obstacle corresponding to the grid. Therefore, for a grid, the probability that it is empty is expressed as p(s=1), and the probability that there is an obstacle is expressed as p(s=0), and the sum of the probabilities of the two is 1. After that, the computing device performs a series of mathematical transformations on each frame of laser point cloud projected to the occupied grid submap, and locates the grid as occupied or idle according to the probability of whether each grid is occupied, wherein the center position of the occupied grid submap is the origin O of the occupied grid submap. Similarly, the above processing is performed for each frame of laser point cloud, so that all n frames of laser point cloud correspond to an occupied grid submap (a total of n), and then the occupied grid submaps corresponding to these n frames of laser point cloud are spliced ​​to obtain a complete OGM.

[0100] For ease of understanding, the following example is used for illustration. Figure 7 , Figure 7 It is illustrated that 9 occupied grid sub-maps constructed corresponding to 9 frames of laser point cloud are spliced ​​into an OGM, where O1 to O9 are the origins of the 9 occupied grid sub-maps, which are spliced ​​in sequence according to the size order of the origin coordinates to obtain a whole OGM. The OGM constitutes a sub-map of the map provided in the embodiment of the present application.

[0101] It should be noted that in some embodiments of the present application, the constructed submap may be an OGM as described in the above related concepts, that is, each occupied grid in the obtained OGM stores the average height of the laser points falling into the grid (that is, the average height of the obstacles in the grid) and the average reflection intensity (that is, the average reflection intensity of the laser points falling into the grid). For details, please refer to Figure 8 , Figure 8 It is shown that a certain occupied grid in the OGM stores the average height h1 (based on the ground) and the average reflection intensity R1 of the corresponding obstacle, where O is the origin of the OGM, that is, the center point.

[0102] It should be noted here that the embodiments of the present application provide Figure 8 The difference between the OGM shown and the existing OGM is that it not only stores the average height of the obstacles corresponding to the occupied grid, but also stores the average reflection intensity of the laser points falling into the occupied grid. The existing OGM only stores the average height of the obstacles, and the reflection intensity of the laser points is stored elsewhere. The advantage of the storage in the embodiment of the present application is that it is easy to find and more convenient in actual application.

[0103] It should also be noted that, in some embodiments of the present application, the constructed sub-map may be an improved OGM, that is, each occupied grid in the obtained OGM can be divided into partially occupied and fully occupied. Full occupation refers to obstacles extending from the ground to a certain height (such as general buildings such as office buildings and residential buildings), and partial occupation refers to buildings such as bridge tunnels, tunnels, viaducts, and aerial crosswalks that occupy a part of the space, which can also be called suspended obstacles. For these two types of occupancy, when an occupied grid in the OGM is fully occupied, that is, the obstacle of the grid is a general obstacle, then the obstacle height stored in the grid is the height of the upper edge of the obstacle from the ground; when an occupied grid in the OGM is partially occupied, that is, the obstacle of the grid is a suspended obstacle, then the obstacle height stored in the grid is the first height of the lower edge of the obstacle from the ground and the second height of the upper edge of the obstacle from the ground. For ease of understanding, please refer to the details. Fig. 9 , Fig. 9 The illustration shows that the two grids in OGM are divided into fully occupied and partially occupied, and the two grids are stored as follows Fig. 9"h2, R2" and "(h0, h3), R3" are shown, where h2 refers to the height value of the obstacle of the corresponding grid extending from the ground to the top of the obstacle, and R2 is the average reflection intensity of the laser points falling into the grid; h0 refers to the first height of the lower edge of the obstacle of the corresponding grid from the ground, and h3 refers to the second height of the upper edge of the obstacle of the corresponding grid from the ground, and R3 is the average reflection intensity of the laser points falling into the grid. It should be noted that in some embodiments of the present application, the OGM provided in the embodiments of the present application can further enrich the storage of obstacle heights. For example, for multi-layer suspended obstacles, height divisions of h01, h02, h03, ..., h0n can also be provided.

[0104] It should be noted here that the embodiments of the present application provide Fig. 9 The difference between the improved OGM shown and the existing OGM is that the obstacles in the occupied grid are classified into general buildings and suspended obstacles, and different heights are stored for different types of obstacles. At the same time, the average reflection intensity of the laser points falling into the occupied grid is also stored, while the existing OGM is considered to be fully occupied if a grid is occupied. The advantage of storing the obstacle height in this way in the embodiment of the present application is that more detailed features of the obstacle are retained, and the accuracy of subsequent positioning is improved.

[0105] It should also be noted that in some embodiments of the present application, the height of the grid obstacle in the OGM can be stored as integer data, the average reflection intensity of the laser point falling into the occupied grid in the OGM can be stored as integer data, and the height of the grid obstacle in the OGM and the average reflection intensity of the laser point falling into the occupied grid can also be stored as integer data. Among them, integer data refers to numerical data that does not contain a decimal part. Integer data is only used to represent integers and is stored in binary form. For example: Taking 0.1m (meter) as an example, int8 data can express a height of 0 to 25.6m. Assuming that the height of an obstacle occupying a grid is 6.7789m, the existing OGM stores the height of the obstacle as a floating point type. The existing OGM will directly store it as a floating point data 6.7789m, and in the embodiment of the present application, it will be stored as an integer data 68, because it is discrete by 0.1m, so the integer data 68 represents 6.8m.

[0106] It should also be noted that, in some implementations of the present application, step 503 may be executed before step 501, step 503 may be executed after step 502, or step 503 may be executed simultaneously with step 501, which is not specifically limited here.

[0107] 504. Create an index of the occupied grid map on the main map.

[0108] After the above processing is performed on each frame of laser point cloud, a main map and an OGM are obtained. Then, it is necessary to link the obtained main map and OGM to form a composite laser map. Specifically, the computing device can establish an index of the OGM on the main map. One implementation method can be: first, the computing device converts the center position (i.e., the origin) of the occupied grid sub-map corresponding to each laser point cloud into a coordinate value in the UTM coordinate system, and then adds each origin coordinate value on the main map as an index label of the occupied grid sub-map corresponding to each frame of laser point cloud.

[0109] For ease of understanding, we still use Figure 7 Take this as an example to illustrate: Figure 7 There are 9 occupancy grid sub-maps, with origins O1 to O9. First, the computing device converts the coordinates of these 9 origins on the grid map into coordinates in the UTM coordinate system (a total of 9 coordinates), and then stores the coordinates of each origin as an index tag in the main map. In actual applications, autonomous vehicles also use this UTM coordinate system to locate themselves.

[0110] In the above implementation of the present application, the computing device performs two operations on each frame of the acquired laser point cloud, such as Fig.10 As shown in the figure, one operation is target feature extraction, constructing a function that fits each target feature, and obtaining the corresponding constraints of each function. These functions and the corresponding constraints of the functions constitute the main map. Another operation is to construct a submap according to each frame of laser point cloud (obtained by splicing the occupied grid submap corresponding to each frame of laser point cloud). The constructed submap is the occupied grid submap (OGM). After that, by establishing the index of OGM on the main map, a composite laser map of the main map and OGM is constructed, which reduces the storage capacity of the composite laser map and retains more feature information for subsequent matching and positioning.

[0111] Based on the composite laser map obtained by the above-mentioned implementation method of this application, the following describes how the autonomous driving vehicle can accurately locate itself based on the composite laser map obtained. Fig.11First, the laser sensor installed on the autonomous driving vehicle acquires the laser point cloud in real time, and pre-processes the laser point cloud acquired at the current moment. At the same time, the IMU on the autonomous driving vehicle senses the position and orientation change of the autonomous driving vehicle and other posture information of the autonomous driving vehicle based on inertial acceleration. Then, based on the composite laser map constructed in advance and combined with the matching algorithm, the positioning result of the autonomous driving vehicle is obtained. The positioning process can be processed in two stages, which can be matched with the main map first, and then matched with the OGM according to the index tag, so as to achieve accurate positioning. General matching algorithms include iterative neighbor point matching, feature matching, filter matching and probability matching. The map directly composed of the original laser point cloud can use a variety of matching algorithms because it retains the original information of the laser point cloud, while the OGM currently used in the industry can generally only perform filter matching and probability matching. The composite laser map constructed by the embodiment of the present application retains both the feature information of the laser point cloud (i.e., the main map) and the gridded point cloud information (i.e., the OGM), so it can simultaneously support the feature matching, filter matching and probability matching of the laser point cloud, and can provide more accurate positioning results on the lightweight composite laser map.

[0112] It should be noted here that when constructing a composite laser map, a standard posture of a standard vehicle will be used to construct it. In the actual positioning process, it is necessary to adjust the initial posture of the autonomous driving vehicle to make it as close to the standard posture as possible, so that the positioning accuracy is higher.

[0113] In the above-mentioned embodiments of the present application, compared with the method of directly constructing the original laser point cloud into a laser map, the composite laser map provided in the embodiment of the present application has lower requirements for storage space and is more lightweight; compared with the existing method of compressing the three-dimensional laser point cloud into two-dimensional information and constructing a two-dimensional OGM based on the compressed two-dimensional information, the composite laser map provided in the embodiment of the present application improves the OGM so that the OGM retains more detailed features. In addition, in some scenes with relatively open areas around roads, the original OGM is relatively uniform. At this time, only using the original OGM for matching often brings relatively large positioning errors (because in different positions in the open scene, some adjacent local positions in the OGM look relatively similar), while the main map in the composite laser map provided in the embodiment of the present application can provide line features and surface features such as street lamp poles and road signs to improve the matching accuracy.

[0114] exist Figure 5 On the basis of the corresponding embodiments, in order to better implement the above solutions of the embodiments of the present application, the following also provides related devices for implementing the above solutions. Fig.12 , Fig.12A structural schematic diagram of a computing device 1200 provided in an embodiment of the present application, the computing device 1200 can be deployed in various intelligent driving (e.g., unmanned driving, assisted driving, etc.) intelligent entities (e.g., autonomous driving vehicles, assisted driving vehicles, etc. in wheeled mobile devices), and is used to construct a composite laser map so that the intelligent entity can perform positioning based on the constructed composite laser map; the computing device 1200 can also be an independent terminal device, such as a mobile phone, personal computer, tablet and other intelligent devices, and is used to construct a composite laser map and send the constructed composite laser map to various intelligent driving (e.g., unmanned driving, assisted driving, etc.) intelligent entities (e.g., autonomous driving vehicles, assisted driving vehicles, etc. in wheeled mobile devices), and is used for positioning the intelligent entity. The computing device 1200 may include: an extraction module 1201, a first construction module 1202, a second construction module 1203 and an indexing module 1204, wherein the extraction module 1201 is used to extract target features based on laser point cloud data, and the target features are laser points that meet preset conditions extracted from the laser point cloud data, and the laser points include the coordinates of the laser points and the reflection intensity of the laser points; the first construction module 1202 is used to construct a function that fits the target features and obtain the constraints of the function, and the function and the constraints constitute a main map; the second construction module 1203 is used to construct an occupancy grid map (OGM) according to the laser point cloud data; the indexing module 1204 is used to establish an index of the OGM on the main map.

[0115] In the above-mentioned embodiment of the present application, the computing device 1200 performs two operations on the acquired laser point cloud. One operation is to extract target features through the extraction module 1201, and further construct a function that fits each target feature through the first construction module 1202, and obtain the restriction conditions corresponding to each function. These functions and the restriction conditions corresponding to the functions constitute the main map. The other operation is to construct a sub-map according to each frame of laser point cloud through the second construction module 1203. The constructed sub-map is OGM. After that, the index of OGM is established on the main map through the index module 1204, and a composite laser map of the main map and OGM is constructed, which reduces the storage capacity of the composite laser map while retaining more feature information for subsequent matching and positioning.

[0116] In a possible design, the second construction module 1203 is specifically used to: construct a first occupied grid submap according to the first laser point cloud data, the first laser point cloud data belongs to any one or more frames in the acquired laser point cloud; and splice the first occupied grid submap corresponding to the constructed first laser point cloud data to obtain the occupied grid map. For example, assuming that there are 100 frames of laser point cloud, then 100 occupied grid submaps can be constructed, and the process of constructing each occupied grid submap is as follows: after setting the length and width of the occupied grid submap (i.e., setting the occupied grid submap size) and the grid resolution, the computing device 1200 projects each frame of the laser point cloud obtained in the laser coordinate system into the corresponding occupied grid submap. If there is no laser point in a grid, it is considered to be empty, and if there is at least one laser point, it is considered that there is an obstacle corresponding to the grid. Therefore, for a grid, the probability that it is empty is represented as p(s=1), and the probability that there is an obstacle is represented as p(s=0), and the sum of the probabilities of the two is 1. After that, the computing device 1200 performs a series of mathematical transformations on each frame of laser point cloud projected to the occupied grid submap, and locates the grid as occupied or idle according to the probability of whether each grid is occupied, wherein the center position of the occupied grid submap is the origin O of the occupied grid submap. Similarly, the above processing is performed for each frame of laser point cloud, so that all n frames of laser point cloud correspond to an occupied grid submap (i.e., the first occupied grid submap, a total of n), and then the occupied grid submaps corresponding to the n frames of laser point cloud are spliced ​​to obtain a complete OGM.

[0117] In the above-mentioned implementation mode of the present application, it is specifically described how the second construction module 1203 constructs corresponding occupancy grid sub-maps from laser point cloud data and stitches these occupancy grid sub-maps into a complete OGM, which is feasible.

[0118] In one possible design, the index module 1204 is specifically used to: first, convert the center position (i.e., the origin) of the occupied grid sub-map corresponding to each laser point cloud into a coordinate value under the UTM coordinate system, and then add each origin coordinate value on the main map as an index label of the occupied grid sub-map corresponding to each frame of laser point cloud.

[0119] In the above implementation of the present application, a specific implementation method of how to establish a connection between the main map and the OGM is explained, that is, adding index tags of each grid sub-map to the main map. This implementation method is easy to implement and simple to operate.

[0120] In one possible design, the first construction module 1202 is specifically used to: obtain the value range of the independent variable corresponding to the function; or, obtain the value of the target independent variable in the function, the target independent variable includes the coordinates of the target laser point, and the target laser point belongs to the target feature.

[0121] In the above-mentioned implementation manner of the present application, specific expressions of the restriction conditions of several functions are given, which are flexible and optional.

[0122] In one possible design, the OGM includes: the height of obstacles in a first grid and the average value of the reflection intensity of laser points falling into the first grid, and the first grid is any occupied grid in the occupied grid map. That is to say, each occupied grid (i.e., the first grid) in the OGM obtained in the embodiment of the present application stores the average height of the laser points falling into the grid (i.e., the average height of the obstacles in the grid) and the average reflection intensity (i.e., the average value of the reflection intensity of the laser points falling into the grid).

[0123] In the above-mentioned embodiment of the present application, the difference between the provided OGM and the existing OGM is that it not only stores the average height of the obstacles corresponding to the occupied grid, but also stores the average reflection intensity of the laser points falling into the occupied grid. The existing OGM only stores the average height of the obstacles, and the reflection intensity of the laser points is stored elsewhere. The advantage of such storage in the embodiment of the present application is that it is easy to search and more convenient in actual application.

[0124] In one possible design, the height of the occupied grid obstacle in the OGM can be stored as integer data, the average reflection intensity of the laser points falling in the occupied grid in the OGM can be stored as integer data, and the height of the occupied grid obstacle in the OGM and the average reflection intensity of the laser points falling in the occupied grid can both be stored as integer data.

[0125] In the above implementation mode of the present application, it is explained that the data occupying the grid storage in the OGM can be integer data. The existing solutions store data in the form of floating-point data. Integer data occupies less storage space than floating-point data (theoretically, the storage capacity occupied by integer data is 1 / 4 of that of floating-point data). Therefore, the advantage of doing so in the implementation mode of the present application is that storage capacity can be saved.

[0126] In a possible design, each occupied grid in the OGM obtained in the embodiment of the present application can be divided into partially occupied and fully occupied. Full occupation refers to obstacles extending from the ground to a certain height (such as general buildings such as office buildings and residential buildings), and partial occupation refers to buildings such as bridge tunnels, tunnels, viaducts, and aerial crosswalks that occupy a part of the space, which can also be called suspended obstacles. For these two types of occupation, when an occupied grid in the OGM is fully occupied, that is, the obstacle of the grid is a general obstacle, then the obstacle height stored in the grid is the height of the upper edge of the obstacle from the ground; when an occupied grid in the OGM is partially occupied, that is, the obstacle of the grid is a suspended obstacle, then the obstacle height stored in the grid is the first height of the lower edge of the obstacle from the ground and the second height of the upper edge of the obstacle from the ground.

[0127] In the above-mentioned implementation of the present application, the difference between the improved OGM provided by the embodiment of the present application and the existing OGM is that the obstacles in the occupied grid are classified into general buildings and suspended obstacles, and different heights are stored for different types of obstacles. At the same time, the average reflection intensity of the laser points falling into the occupied grid is also stored, while the existing OGM is considered to be completely occupied if a grid is occupied. The advantage of storing the height of obstacles in this way in the embodiment of the present application is that more detailed features of the obstacles are retained, thereby improving the accuracy of subsequent positioning.

[0128] In one possible design, the target features of the embodiment of the present application are essentially to extract some special laser points from a frame of laser point cloud. The extracted target features are generally line features and surface features, where the line features are used to indicate that the laser points extracted from the laser point cloud data are located on the same straight line, and the surface features are used to indicate that the laser points extracted from the laser point cloud data are located on the same plane.

[0129] In the above-mentioned embodiments of the present application, some conditions that the extracted target features meet are specifically described, so that the extracted target features have key features that are conducive to subsequent positioning.

[0130] It should be noted that Fig.12 The information interaction, execution process, etc. between the modules / units in the computing device described in the corresponding embodiment are the same as those in the present application. Figure 5 The corresponding method embodiments are based on the same concept. For specific contents, please refer to the description in the method embodiments shown above in this application, which will not be repeated here.

[0131] The present application also provides a computing device. Fig.13 , Fig.13This is a schematic diagram of a structure of a computing device provided by an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computing device 1300 may be deployed with Fig.12 The module of the computing device described in the corresponding embodiment is used to implement Fig.12 The functions of the computing device in the corresponding embodiment, specifically, the computing device 1300 is implemented by one or more servers, and the computing device 1300 may have relatively large differences due to different configurations or performances, and may include one or more central processing units (CPU) 1322 (for example, one or more) and memory 1332, one or more storage media 1330 (for example, one or more mass storage devices) storing application programs 1342 or data 1344. Among them, the memory 1332 and the storage medium 1330 can be short-term storage or persistent storage. The program stored in the storage medium 1330 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the training device. Furthermore, the central processing unit 1322 can be configured to communicate with the storage medium 1330 to execute a series of instruction operations in the storage medium 1330 on the computing device 1300.

[0132] The computing device 1300 may also include one or more power supplies 1326, one or more wired or wireless network interfaces 1350, one or more input and output interfaces 1358, and / or one or more operating systems 1341, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0133] In the embodiment of the present application, the above Figure 5 The steps performed by the computing device in the corresponding embodiment may be based on the Fig.13 The structure shown is implemented and will not be described in detail here.

[0134] It should also be noted that the device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed over multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present application, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines.

[0135] Through the description of the above implementation mode, the technicians in the relevant field can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course, it can also be implemented by special hardware including special integrated circuits, special CPUs, special memories, special components, etc. In general, all functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits or special circuits. However, for the present application, software program implementation is a better implementation mode in more cases. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer floppy disk, a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a training device, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0136] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0137] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, training equipment or data center to another website site, computer, training equipment or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device, data center, etc. that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a high-density digital video disc (digital video disc, DVD)), or a semiconductor medium (e.g., a solid-state drive (solid statedisk, SSD)), etc.

Claims

1. A method for constructing a map, characterized in that: include: Extracting target features based on the laser point cloud data, wherein the target features are laser points that meet preset conditions and are extracted from the laser point cloud data, and the laser points include the coordinates of the laser points and the reflection intensity of the laser points; Constructing a function that fits the target feature and obtaining a restriction condition of the function, wherein the function and the restriction condition constitute a main map; constructing an occupancy grid map based on the laser point cloud data; An index of the occupancy grid map is established on the main map, and the establishment method includes: converting the center position of the occupancy grid sub-map corresponding to each frame of laser point cloud into a coordinate value in a Universal Transverse Mercator Grid (UTM) coordinate system, and adding the coordinate value on the main map as an index label of the occupancy grid sub-map corresponding to each frame of laser point cloud, and the occupancy grid sub-map belongs to the occupancy grid map.

2. The method according to claim 1, characterized in that The constructing an occupancy grid map according to the laser point cloud data comprises: constructing a first occupancy grid sub-map according to first laser point cloud data, wherein the first laser point cloud data belongs to any one or more frames of the laser point cloud data; The constructed first occupancy grid sub-maps are spliced ​​to obtain the occupancy grid map.

3. The method according to claim 2, characterized in that The step of establishing the index of the occupation grid map on the main map comprises: Convert the center position of the constructed first occupied grid sub-map into a coordinate value in the UTM coordinate system; Add the coordinate value as an index tag of the constructed first occupancy grid sub-map on the main map.

4. The method according to claim 1, characterized in that: The restriction conditions for obtaining the function include: Obtaining a value interval of an independent variable corresponding to the function; or, The value of the target independent variable in the function is obtained, wherein the target independent variable includes the coordinates of the target laser point, and the target laser point belongs to the target feature.

5. The method according to any one of claims 1 to 4, characterized in that The occupancy grid map includes: The height of the obstacle in the first grid and the average value of the reflection intensity of the laser point falling into the first grid, and the first grid is any occupied grid in the occupied grid map.

6. The method according to claim 5, characterized in that The height of the obstacle in the first grid is stored as integer data; and / or, The average value of the reflection intensity of the laser points falling within the first grid is stored as the shaping data.

7. The method according to claim 5, characterized in that When the obstacle in the first grid is a suspended obstacle, the height of the obstacle includes: The lower edge of the obstacle is at a first height from the ground, and the upper edge of the obstacle is at a second height from the ground.

8. The method according to any one of claims 1 to 4, characterized in that: The target features include: A line feature, wherein the line feature is used to indicate that the laser points extracted from the laser point cloud data are located on the same straight line; and / or, A surface feature is used to indicate that the laser points extracted from the laser point cloud data are located on the same plane.

9. A computing device, characterized in that include: An extraction module extracts target features based on the laser point cloud data, wherein the target features are laser points that meet preset conditions and are extracted from the laser point cloud data, and the laser points include the coordinates of the laser points and the reflection intensity of the laser points; A first construction module is used to construct a function that fits the target feature and obtain a restriction condition of the function, wherein the function and the restriction condition constitute a main map; A second construction module is used to construct an occupancy grid map according to the laser point cloud data; An indexing module is used to establish an index of the occupancy grid map on the main map, wherein the establishment method includes: converting the center position of the occupancy grid sub-map corresponding to each frame of laser point cloud into a coordinate value in a Universal Transverse Mercator Grid (UTM) coordinate system, and adding the coordinate value as an index tag of the occupancy grid sub-map corresponding to each frame of laser point cloud on the main map, and the occupancy grid sub-map belongs to the occupancy grid map.

10. The device according to claim 9, characterized in that The second building block is specifically used for: constructing a first occupancy grid sub-map according to first laser point cloud data, wherein the first laser point cloud data belongs to any one or more frames of the laser point cloud data; The constructed first occupancy grid sub-maps are spliced ​​to obtain the occupancy grid map.

11. The device according to claim 10, characterized in that The index module is specifically used for: Convert the center position of the constructed first occupied grid submap into a coordinate value in a Universal Transverse Mercator Grid (UTM) coordinate system; Add the coordinate value as an index tag of the constructed first occupancy grid sub-map on the main map.

12. The device according to claim 9, characterized in that The first building module is specifically used for: Obtaining a value interval of an independent variable corresponding to the function; or, The value of the target independent variable in the function is obtained, wherein the target independent variable includes the coordinates of the target laser point, and the target laser point belongs to the target feature.

13. The device according to any one of claims 9 to 12, characterized in that The occupancy grid map includes: The height of the obstacle in the first grid and the average value of the reflection intensity of the laser point falling into the first grid, and the first grid is any occupied grid in the occupied grid map.

14. The device according to claim 13, characterized in that The height of the obstacle in the first grid is stored as integer data; and / or, The average value of the reflection intensity of the laser points falling within the first grid is stored as the shaping data.

15. The device according to claim 13, characterized in that When the obstacle in the first grid is a suspended obstacle, the height of the obstacle includes: The lower edge of the obstacle is at a first height from the ground, and the upper edge of the obstacle is at a second height from the ground.

16. The device according to any one of claims 9 to 12, characterized in that The target features include: A line feature, wherein the line feature is used to indicate that the laser points extracted from the laser point cloud data are located on the same straight line; and / or, A surface feature is used to indicate that the laser points extracted from the laser point cloud data are located on the same plane.

17. A computing device, characterized in that The method comprises a processor, wherein the processor is coupled to a memory, wherein the memory stores program instructions, and when the program instructions stored in the memory are executed by the processor, the method according to any one of claims 1 to 8 is implemented.

18. A chip system, characterized in that: The chip system includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a computer program or instruction so that the method described in any one of claims 1 to 8 is executed.

19. A computer-readable storage medium comprising a program, which, when executed on a computer, causes the computer to execute the method according to any one of claims 1 to 8.

Images