Data processing method and device, automatic driving system, electronic equipment and storage medium

Abstract

The invention provides a data processing method and device, an automatic driving system, electronic equipment and a storage medium, and belongs to the technical field of data processing.The method comprises the steps that point cloud data output by a laser radar are received, and all points contain three-dimensional coordinate information under a laser radar coordinate system; according to preset grid map parameters, performing grid mapping operation on each point converted to the target coordinate system so as to determine a grid unit to which each point belongs; based on the grid units determined by all the point cloud data points, a local grid map is obtained, the local grid map marks the vehicle surrounding environment state in a discretization mode, and each grid unit marks the environment state within the preset range of the grid unit. The local grid map provided by the invention provides a basis for quickly and accurately identifying obstacles and generating reliable path planning.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and specifically to a data processing method, device, automatic driving system, electronic device and storage medium. Background Art

[0002] Autonomous driving technology integrates advanced technologies in multiple fields such as artificial intelligence, robot perception and decision-making, computer vision, deep learning, parallel computing and hardware acceleration, and realizes real-time perception and intelligent decision-making of the vehicle's surrounding environment through high-precision point cloud data from lidar. However, the main problems currently faced by autonomous driving technology include: limited perception performance in complex environments (such as nighttime and bad weather), challenges in real-time and stability of data processing, complexity and high computing resource requirements of multi-sensor fusion algorithms, and limited generalization capabilities and practical application promotion of deep learning end-to-end driving systems. Summary of the invention

[0003] The present application provides a data processing method, device, autonomous driving system, electronic device and storage medium, aiming to solve the problem of how to quickly and accurately identify obstacles in a dynamic and complex environment to generate reliable path planning.

[0004] In a first aspect, the present application provides a data processing method, the method comprising:

[0005] Receive point cloud data output by the laser radar, where each point contains three-dimensional coordinate information in the laser radar coordinate system;

[0006] According to the preset grid map parameters, a grid mapping operation is performed on each point converted to the target coordinate system to determine the grid unit to which each point belongs;

[0007] Based on the grid cells determined by all the point cloud data points, a local grid map is obtained, and the local grid map marks the state of the vehicle's surrounding environment in a discrete form, and each grid cell marks the environment state within its preset range.

[0008] In a second aspect, the present application further provides a data processing device, the device comprising:

[0009] A data receiving module is used to receive point cloud data output by the laser radar, wherein each point contains three-dimensional coordinate information in the laser radar coordinate system;

[0010] A grid mapping module is used to perform a grid mapping operation on each point converted to the target coordinate system according to preset grid map parameters to determine the grid unit to which each point belongs;

[0011] The map generation module is used to obtain a local grid map based on the grid cells determined by all point cloud data points. The local grid map marks the state of the vehicle's surrounding environment in a discrete form, and each grid cell marks the environment state within its preset range.

[0012] In a third aspect, the present application provides an autonomous driving system, the system comprising:

[0013] A perception module, which includes a laser radar and is used to collect point cloud data of the surrounding environment in real time;

[0014] A data processing module, used to execute the steps of the data processing method as described in the first aspect;

[0015] The control module is used for path planning, obstacle detection and obstacle avoidance control based on the local grid map and the real-time status information of the vehicle.

[0016] In a fourth aspect, the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, the steps of the data processing method as described in any one of the first aspects are implemented.

[0017] In a fifth aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the data processing method as described in any one of the first aspects are implemented.

[0018] The present application provides a data processing method, device, automatic driving system, electronic device and storage medium, which aims to solve the problem of how to quickly and accurately identify obstacles in a dynamic and complex environment to generate reliable path planning. The method first receives the point cloud data output by the laser radar to obtain each point with three-dimensional coordinate information in the laser radar coordinate system. The laser radar can scan the surrounding environment, and the point cloud data is a discretized representation of the environment, which contains rich environmental information. Then, according to the preset grid map parameters, the point cloud data is converted to the target coordinate system, and the grid mapping operation is performed on each point to correspond the points in the point cloud data to the grid cells. The continuous point cloud data is converted into a discretized grid representation. Finally, a local grid map is obtained based on the grid cells determined by all the point cloud data points. The local grid map marks the state of the vehicle's surrounding environment in a discretized form, and each grid cell marks the state of the environment within its preset range, so that different environmental elements can be identified, such as possible obstacles. After the obstacles are identified, in the path planning algorithm, these grid cells marked as obstacles can be avoided to generate a reliable path planning.

[0019] Therefore, this application constructs a local grid map based on the grid cells determined by all point cloud data points, and the local grid map marks the state of the vehicle's surrounding environment in a discrete form. In a dynamic and complex environment, the local grid map provides a basis for quickly and accurately identifying obstacles and generating reliable path planning. With the help of marking the environmental state of each grid cell, grid cells with obstacles can be easily identified, and these areas can be avoided during path planning. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0021] Figure 1 is a flow chart of the data processing method provided by the present application;

[0022] Figure 2 It is a structural schematic diagram of a data processing device provided by the present application;

[0023] Figure 3 is a schematic diagram of the structure of the autonomous driving system provided by this application;

[0024] Figure 4 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein.

[0027] In order to solve the problem of how to quickly and accurately identify obstacles in a dynamic and complex environment to generate a reliable path planning, the present application provides a data processing method, device, automatic driving system, electronic device and storage medium. The method receives the point cloud data of the lidar, converts the coordinate system according to the preset grid map parameters and performs a grid mapping operation, corresponds the points to the grid cells, and constructs a local grid map based on these grid cells. The local grid map marks the state of the vehicle's surrounding environment in a discrete form and can identify environmental elements including obstacles. In a dynamic and complex environment, it provides a basis for quickly and accurately identifying obstacles and generating reliable path planning, and grid cells with obstacles can be avoided during path planning.

[0028] Combine the following Figure 1-Figure 4 The present application describes a data processing method, device, autonomous driving system, electronic device, and storage medium.

[0029] Please refer to Figure 1 , Figure 1 The data processing method provided by the present application is a flowchart. A data processing method comprises the following steps:

[0030] S110, receiving point cloud data output by the laser radar, wherein each point includes three-dimensional coordinate information in the laser radar coordinate system.

[0031] Specifically, in self-driving cars, LiDAR is a key sensor. It can sense the environment around the vehicle in real time, including the location and shape information of roads, other vehicles, pedestrians, traffic signs, etc., providing an important basis for vehicle path planning, obstacle avoidance, etc. The LiDAR coordinate system is a specific coordinate system that is established based on the installation position and measurement direction of the LiDAR itself. The three-dimensional coordinate information represents the position of each point in this coordinate system, which can be expressed as (x, y, z), where x, y, and z represent the coordinate values ​​in the directions of the three coordinate axes, respectively.

[0032] S120, performing a grid mapping operation on each point converted to the target coordinate system according to preset grid map parameters to determine the grid unit to which each point belongs.

[0033] Specifically, this step specifically includes preprocessing, coordinate system conversion and rasterization steps. Point cloud data is a three-dimensional data set about the surrounding environment obtained by laser radar. However, in the process of collecting data, laser radar will inevitably introduce noise, and there may also be some outliers (such as data points that deviate greatly from the actual environment due to reflection anomalies, etc.). Therefore, it is necessary to use filtering technology to preprocess the point cloud data. The present application provides two filtering technologies, Gaussian filtering and median filtering. Among them, Gaussian filtering is a weighted filtering method based on normal distribution. It assumes that the noise conforms to the Gaussian distribution, and smoothes the data by weighted averaging each data point and the data points in its neighborhood according to a certain weight, so as to achieve the purpose of removing noise. For example, for a point in a point cloud data, the points around it are given different weights according to the distance from the point. The closer the distance, the greater the weight, and then the weighted average is calculated to replace the original value of the point, so that the data can be smoother. The median filter is a nonlinear signal processing technology based on sorting statistics theory. It sorts the values ​​of a point and the points in its neighborhood, and then takes the middle value as the new value of the point. After filtering, the result is point cloud data that has been freed of interfering factors such as sensor noise and outliers.

[0034] Convert the preprocessed point cloud data from the LiDAR coordinate system to the target coordinate system (such as the vehicle or map coordinate system). The LiDAR coordinate system is established based on the installation position and measurement direction of the LiDAR itself. However, in autonomous driving or other application scenarios, in order to facilitate operations such as fusion with other sensor data of the vehicle, path planning, or matching with map information, it is necessary to convert the point cloud data to the target coordinate system (such as the vehicle coordinate system, that is, the coordinate system with the vehicle itself as the reference center, or the map coordinate system, that is, the coordinate system with the entire map as the reference). The result of the conversion is that the point cloud data is aligned with the target position. For example, if in the vehicle coordinate system, the center position of the vehicle is the coordinate origin, then the coordinates of the points in the converted point cloud data are relative to the center position of the vehicle.

[0035] In the rasterization process, it is necessary to determine the preset raster map parameters, which include resolution and range. Among them, the resolution determines the fineness of the raster. For example, each raster represents an environmental area of ​​a certain size, such as 1 meter × 1 meter, which is a resolution setting. If the resolution is higher, such as 0.5 meters × 0.5 meters, the raster will be smaller, and the representation of the environment will be more detailed, but it will also increase the amount of data and computing costs; conversely, if the resolution is lower, such as 2 meters × 2 meters, the raster is larger, although the amount of data and computing costs will be reduced, but the description of environmental details will be coarser. The range determines the size of the area to be rasterized. The range can be set according to actual needs. For example, in an autonomous driving scenario, it can be an area within a certain radius around the vehicle. When the resolution and range are determined, each point in the point cloud data is mapped to a fixed-size grid. The point cloud data contains information such as the coordinates of each point in the environment, and the points are classified into corresponding grid cells according to their coordinates. For example, if the coordinates of a point are within a 1 meter by 1 meter grid, then the point is mapped to this grid cell.

[0036] S130, based on the grid cells determined by all the point cloud data points, a local grid map is obtained, wherein the local grid map marks the state of the environment around the vehicle in a discrete form, and each grid cell marks the state of the environment within a preset range.

[0037] Specifically, after the rasterization operation, a discrete local grid map is obtained. Different from the original point cloud data (which is continuous and composed of many discrete points), the local grid map is discrete, and the local grid map divides the environment into regular grid cells. In the local grid map, each grid cell represents a certain environmental state. The environmental state can contain a variety of information, such as whether there are obstacles, whether they are static obstacles or dynamic obstacles, etc. This information can be obtained by analyzing the point cloud data mapped to the grid cells.

[0038] Exemplarily, the method further includes assigning different numerical values ​​to each grid cell to mark an environmental state, wherein the environmental state includes a static obstacle, a dynamic obstacle, and no obstacle;

[0039] If there is a static obstacle (such as a building, a fixed roadblock, etc.) in the grid cell, a first value (such as 1) is assigned; if there is a dynamic obstacle (such as a moving vehicle, a moving pedestrian, etc.) in the grid cell, a second value (such as 2) is assigned; if there is no obstacle (i.e., a passable area) in the grid cell, a third value (such as 0) is assigned. The first value, the second value, and the third value may be different.

[0040] Finally, the output local grid map facilitates environmental analysis and decision-making. In terms of environmental analysis, the layout of the environment can be viewed intuitively, for example, by viewing the status of the grid cells to determine where there are obstacles and where the passable area is. In terms of decision-making, for example, in autonomous driving, the vehicle's driving path can be planned based on the local grid map, avoiding grid cells with obstacles and choosing the best path to the target; or in robot navigation, the robot's moving direction can be determined.

[0041] The above steps S110 to S130 are described in detail below.

[0042] When processing a large amount of point cloud data, whether querying the location information of a certain point or performing a rasterization operation (i.e. mapping the point cloud data to a grid cell), if all the point cloud data are directly traversed, when the amount of data is large, the amount of calculation will be very large and the efficiency will be very low. Therefore, this application provides a spatial index structure that can quickly process a large amount of point cloud data.

[0043] In some embodiments, the data processing method described in the present application further includes:

[0044] S140, constructing a spatial index structure, where the spatial index structure is used to map the point cloud data to corresponding grid cells.

[0045] Specifically, the construction of the spatial index structure provides support for the grid mapping operation in step S120. Through the spatial index structure, points in the point cloud data can be more efficiently found, so that the points can be accurately mapped to the corresponding grid cells. And by accurately determining the grid cell to which each point cloud data point belongs, an accurate local grid map can be obtained.

[0046] Exemplarily, step S140 specifically includes S141 to S143:

[0047] S141, selecting a coordinate axis in the target coordinate system as an initial segmentation axis, and performing a sorting operation on all point cloud data according to the initial segmentation axis to segment them into two subsets.

[0048] For example, assuming the point cloud data in the same target coordinate system is:

[0049] (1,2,3), (2,3,4), (3,4,5), (4,5,6), (5,6,7), (6,7,8), (7,8,9), (8,9,10), (9,10,11), (10,11,12), and the preset threshold is set to 1.

[0050] If the x-axis is selected as the initial segmentation axis, the sorting operation is performed on all point cloud data according to the x-axis, and the sorted result is:

[0051] (1,2,3), (2,3,4), (3,4,5), (4,5,6), (5,6,7), (6,7,8), (7,8,9), (8,9,10), (9,10,11), (10,11,12).

[0052] Then split it into two subsets, which can be split according to the middle point (such as the fifth point (5,6,7)) to obtain two subsets:

[0053] Subset 1: (1,2,3), (2,3,4), (3,4,5), (4,5,6);

[0054] Subset 2: (6,7,8), (7,8,9), (8,9,10), (9,10,11), (10,11,12).

[0055] S142, for the two subsets that have been segmented, a construction operation is performed recursively, including selecting the next coordinate axis of the target coordinate system as the next segmentation axis in each subset, and performing a sorting operation on the point cloud data points in the corresponding subset based on the next segmentation axis.

[0056] For example, for subset 1: select the next coordinate axis (y axis) of the target coordinate system as the next split axis. Sorting is performed on the point cloud data points in subset 1 based on the y axis. The sorted results are: (1,2,3), (2,3,4), (3,4,5), (4,5,6). Then, according to the middle point (such as the second point (2,3,4)), subset 1 is split into two smaller subsets:

[0057] Subset 1-1: (1,2,3);

[0058] Subset 1-2: (3,4,5), (4,5,6).

[0059] For subset 2: the y-axis is also selected as the next split axis, and the point cloud data points in subset 2 are sorted based on the y-axis. The sorted results are: (6,7,8), (7,8,9), (8,9,10), (9,10,11), (10,11,12). Subset 2 is split into two smaller subsets based on the middle point (such as the third point (8,9,10)):

[0060] Subset 2-1: (6,7,8), (7,8,9);

[0061] Subset 2-2: (9, 10, 11), (10, 11, 12).

[0062] S143, and so on, until the number of points included in the subset is less than or equal to the preset threshold, a constructed spatial index structure is obtained.

[0063] For example, for subset 1-1, since it contains only one point and the number of points is equal to the preset threshold 1, no further segmentation is performed. For subset 1-2, the z axis is selected as the segmentation axis (because the x axis and y axis have been used), and after sorting, the segmentation is performed according to the middle point, and the recursive operation is continued until the number of points contained in the subset is less than or equal to the preset threshold. Similar operations are performed for subsets 2-1 and 2-2 until the number of points contained in all subsets is less than or equal to the preset threshold, thus obtaining a constructed spatial index structure.

[0064] The constructed spatial index structure can map point cloud data to corresponding grid cells. For example, suppose there is a grid map, and the range of each grid cell in three-dimensional space is 1×1×1. If you want to determine which grid cell the point (3,4,5) belongs to, you can use this spatial index structure to quickly locate the area where the point may be located, and then determine the grid cell to which it belongs, without traversing all the point cloud data.

[0065] In some embodiments, the method further comprises:

[0066] S150, using the spatial index structure, searching for points located in a specific area in the point cloud data to implement range query and nearest neighbor query, wherein the range query is to find all points in the specific area, and the nearest neighbor query is to find the point closest to a specific point.

[0067] Specifically, when generating a local grid map, it is first necessary to determine which point cloud data is related to the local area to be constructed. Range query can accurately find all point cloud data points in a specific area. For example, in an autonomous driving scenario, if you want to build a local grid map of a specific area in front of the vehicle, you can filter out point cloud data located in the specific area in front (such as a rectangular area of ​​10 meters in front, 5 meters on each side, and 3 meters in height) through range query. This helps to reduce unnecessary data processing, focus only on data that is useful for building a local grid map, and improve computing efficiency. Without range query, all point cloud data needs to be processed, which consumes a lot of computing resources, especially in the case of large-scale point cloud data. In addition, the point cloud data point set obtained by the range query directly determines the content of the local grid map. These points will be mapped to grid cells to construct a local grid map. For example, if the point cloud data obtained by the range query contains part of the point cloud of the road surface, roadside buildings, and other vehicles, then after these points are mapped to the grid cells, information such as roads, building outlines, and vehicle locations will be presented in the local grid map. At the same time, the specific area of ​​the range query also defines the boundaries of the local grid map to a certain extent. For example, the size and shape of the specific area (such as a rectangular area) will correspond to the range and shape of the local grid map.

[0068] When constructing a local raster map, the nearest neighbor query helps to more accurately represent the objects in the map. For example, when there are multiple discrete points in the point cloud data that may belong to the same object (such as an obstacle of complex shape), the nearest neighbor query can be used to find the point closest to a point, and these points can be combined to more accurately depict the shape of the object. Assuming that there is a suspected obstacle represented as multiple discrete point clouds in the local raster map, the nearest neighbor query can be used to find the adjacent relationship between these points, so that they can be merged into a more continuous and accurate object representation, thereby improving the accuracy of the local raster map's description of the environment. In the process of generating a local raster map, some special situations may be encountered, such as the uneven distribution of some point cloud data or the presence of noise points. The nearest neighbor query can help identify and handle these situations. For example, if there is an isolated noise point that is relatively far away from the surrounding points, the nearest neighbor query can distinguish it from the normal point cloud data, avoiding the adverse effects of this noise point on the construction of the local raster map.

[0069] Nearest neighbor queries can also be used to optimize the construction of local grid maps. For example, when determining the properties of a grid cell (such as whether it is occupied, free, or partially occupied), a more reasonable judgment can be made based on the situation of the nearest neighbor points. If most of the nearest neighbors near a grid cell belong to an object, then the grid cell is more likely to be judged as occupied by the object.

[0070] In some embodiments, in the above step S150, the step of using the spatial index structure to search for points located in a specific area in the point cloud data to implement range query and nearest neighbor query includes:

[0071] S121, when performing a grid mapping operation, for a specific area corresponding to each grid cell, a range query is used to locate points that may be within the specific area.

[0072] For example, suppose you want to build a local grid map of the environment around an autonomous vehicle. The vehicle is located at the coordinate origin, and the area 10 meters in front of the vehicle, 5 meters on each side, and 3 meters in height is divided into the range of the local grid map. Each grid cell is a cube with a side length of 0.5 meters. Consider one of the grid cells, whose center coordinates in the target coordinate system are (2,1,1). The specific area corresponding to this grid cell is a cube space with a side length of 0.5 meters and a center of (2,1,1). Use the constructed spatial index structure to manage point cloud data, and use range queries for the specific area of ​​this grid cell when performing raster mapping operations. The details are as follows:

[0073] First, start from the root node of the spatial index structure, which represents the point cloud data space around the entire vehicle. For example, the spatial range of the root node is the space 15 meters in front of the vehicle, 7 meters on each side, and 4 meters in height. Then, compare the relationship between the space represented by the root node and the specific area of ​​the grid unit. Since the root node space contains the specific area, it is necessary to further check the child nodes of the root node. Assume that the root node is split for the first time according to the x-axis, the left child node represents the space 0-7.5 meters in front of the vehicle, and the right child node represents the space 7.5-15 meters in front of the vehicle. The space of the left child node contains the specific area, and continue to check the child nodes of the left child node (split according to the y-axis). Assume that the left child node of the left child node (after splitting according to the y-axis) represents the space 3.5-0 meters on the left side of the vehicle, and this space has an intersection with the specific area on the y-axis, and continue to check the child nodes of this child node (split according to the z-axis). Assume that this left child node after the z-axis split represents the space 0-2 meters in height, and this space has an intersection with the specific area on the z-axis. Through such a recursive search, the point cloud data points within the specific area of ​​this grid unit can be found eventually.

[0074] S122, when it is necessary to determine whether a certain point belongs to a certain grid cell, and the point is close to the grid cell boundary, a nearest neighbor query is used to find the point closest to the point to determine whether the point belongs to the current grid cell.

[0075] For example, in the scenario of an autonomous vehicle constructing a local grid map, assume there is a point P with coordinates (1.9, 1.2, 1.1), and this point is close to the boundary of a grid cell. The center coordinates of this grid cell are (2, 1, 1), and the side length is 0.5 meters. To determine whether point P belongs to this grid cell, a nearest neighbor query can be used. Specifically as follows:

[0076] First, start the query from the spatial index structure. Calculate the distance d1 from point P to a representative point (such as the point at the center of the space) in the space represented by the root node of the spatial index structure, and at the same time record the currently found nearest distance as a relatively large value (initialized to infinity). Assume that the root node is first divided along the x-axis, the left child node represents the space from 0 to 7.5 meters in front of the vehicle, and the right child node represents the space from 7.5 to 15 meters in front of the vehicle. Calculate the distance d2 from the representative point in the left child node space to point P, and the distance d3 from the representative point in the right child node space to point P. If d2 < d3, then first search the left subtree. During the search of the left subtree, if it is found that the distance d4 from a certain point to P is less than the currently recorded nearest distance, then update the nearest distance to d4. Assume that during the search, the point Q with the closest distance to point P is found, and its coordinates are (2.05, 1.1, 1.05). By comparing the relationship between point P and point Q and the grid cell, it can be determined whether point P belongs to the current grid cell. For example, if point Q belongs to this grid cell, and the distance from point P to point Q is within a certain range (according to the definition of the grid cell and the allowable error range), then it can be determined that point P also belongs to this grid cell.

[0077] In some embodiments, the method further includes:

[0078] S160. Generate a path plan according to the local grid map, which specifically includes:

[0079] S1611. Set the current position of the vehicle as the starting point, and at the same time set the end position that the vehicle is to reach as the target point, and use the starting point as the starting node to search all possible adjacent nodes of the starting node.

[0080] For example, in an autonomous driving scenario, assume that the vehicle is driving in an urban road environment and wants to go from its current location to a set destination (such as a specific parking lot). The local grid map represents the environment around the vehicle, including roads, buildings, other vehicles, and pedestrians. The current location of the vehicle is accurately determined by GPS and on-board sensors at a certain location on a certain street, and the grid corresponding to this location is set as the starting point in the local grid map, that is, the starting node. The destination that the vehicle wants to reach is the grid corresponding to a certain entrance location of a specific parking lot. The adjacent nodes of the starting node are the surrounding grids that the vehicle can directly reach from the current location. For example, if the vehicle is on a three-lane road and is currently in the middle lane, then its adjacent nodes may be the grids in the same lane or adjacent lane in front, left front, right front (assuming the vehicle can change lanes), and may also be the grids in the roadside parking areas on the left and right sides (if the vehicle can enter).

[0081] S1612: For each adjacent node, calculate a total cost from the starting point to the target point, where the total cost is the sum of a first cost from the starting point to the adjacent node and a second cost from the adjacent node to the target point.

[0082] For example, taking an adjacent node of a vehicle's starting node as an example, it is assumed that this adjacent node is a grid in the same lane in front of the vehicle.

[0083] (1) The first cost calculation from the starting point to this adjacent node:

[0084] Distance cost: If the distance between two grids is 1 meter, the distance cost can be set to be proportional to the distance, for example, assuming it is 1, depending on the vehicle's speed and driving strategy.

[0085] Safety cost: If there are other vehicles or pedestrians near this adjacent grid, the safety cost will increase. Assuming there is a slow-moving vehicle nearby, the safety cost will increase by 2.

[0086] Traffic rule cost: If you need to follow special traffic rules (such as speed bumps, traffic signs, etc.) in this grid area, the cost will also increase. Assuming there is a speed bump here, the traffic rule cost increases by 1. So the first cost from the starting point to this adjacent node is 1+2+1=4.

[0087] (2) The second cost calculation from this adjacent node to the target point (parking lot entrance):

[0088] First, calculate the approximate distance from this adjacent node to the target point, which can be estimated through map data and path planning algorithms. Assume that the cost corresponding to this distance is 100 (taking into account possible road twists and turns, traffic conditions, etc.). Also consider the traffic conditions that may be encountered from this adjacent node to the target point. For example, you may pass through a congested intersection, and this congestion will increase the cost. Assume that the congestion cost increases by 20. So the second cost from this adjacent node to the target point is 100+20=120. Then the total cost from the starting point to this adjacent node is 4+120=124.

[0089] Similarly, the total cost from other adjacent nodes of the starting node (such as the left front grid, the right front grid, etc.) to the target point is calculated.

[0090] S1613, marking the adjacent node with the smallest total cost as the current node, continuing to search for the adjacent nodes of the current node, and repeating the total cost calculation operation until the target point is reached, connecting the path nodes from the starting point to the target point to generate a path plan; wherein, during the search process, the grid marked as an obstacle in the local grid map is not regarded as an extendable path node.

[0091] For example, based on the total cost calculated previously, assume that the total cost of the right front grid is the smallest. For example, the total cost of the right front grid is 100 (the calculation process is similar to the above), and this right front grid is marked as the current node. Then continue searching for the adjacent nodes of the current node (right front grid). For example, the adjacent node of the right front grid may be the right front grid further ahead, the front grid of the right lane, etc. Calculate the total cost from the starting point through the right front grid to its adjacent node (such as the front grid of the right lane).

[0092] (1) The cost from the starting point to the right front grid has been calculated as part (assuming it is 30). The first cost from the right front grid to the front grid of the right lane is:

[0093] Distance cost: Assumed to be 1.

[0094] Safety cost: If there is a certain risk in the lane change operation (such as a fast-moving vehicle on the side), the safety cost increases by 3.

[0095] Traffic rule cost: If lane change requires certain conditions (such as changing lanes at the dotted line), it is assumed that there is no additional cost. Therefore, the first cost from the right front grid to the front grid of the right lane is 1+3=4.

[0096] (2) The second cost from the front grid of the right lane to the target point (parking lot entrance): Assumed to be 90 (taking into account the new path conditions, possible traffic conditions, etc.). Then the total cost from the starting point through the right front grid to the front grid of the right lane is 30+4+90=124.

[0097] During the search process, if a grid marked as an obstacle is encountered in the local grid map, for example, a grid occupied by a faulty vehicle in the middle of the road is marked as an obstacle, then when searching for adjacent nodes from the current node, the grid occupied by the obstacle will not be considered as an extendable path node. This process continues until the target point (the grid corresponding to the parking lot entrance) is reached, and finally the path nodes from the starting point (the grid corresponding to the initial position of the vehicle) to the target point are connected, thus generating a path plan for the autonomous vehicle from the current position to the destination in an urban road environment.

[0098] In some embodiments, the method further comprises:

[0099] S170: Dynamically update the local grid map.

[0100] Specifically, in the autonomous driving scenario, the environment around the vehicle is constantly changing. For example, while the vehicle is driving, other vehicles may approach or leave, pedestrians may walk on the road, or new obstacles may appear on the roadside (such as temporarily parked vehicles, fallen objects, etc.). Dynamically updating the local grid map can reflect these changes in a timely manner. The decision-making of the autonomous driving system (such as path planning, speed control, etc.) depends on accurate perception of the environment. The dynamically updated local grid map provides the latest environmental information, making decisions more accurate.

[0101] Exemplarily, the above step S170 includes steps S171 to S173:

[0102] S171, adding newly received point cloud data, which includes: for the newly received point cloud data, mapping each point therein to the corresponding grid cell; if the grid cell is empty, adding the information of the new point to the grid cell; if the grid cell already has data, fusing the new point information with the existing information.

[0103] For example, the coordinates of a point in the newly received point cloud data are (3.5, 4.2). Map this point to the corresponding grid cell. Assuming that the side length of each grid cell is 1 meter, this point will be mapped to the grid cell with coordinates (3, 4). If this grid cell is empty (that is, no point cloud data has been received and mapped to this grid cell before), add the information of this new point (such as the distance from the point to the laser radar, reflection intensity, etc.) to the grid cell. Assuming that the coordinates of another point in the newly received point cloud data are (3.2, 4.1), it will also be mapped to the grid cell with coordinates (3, 4). At this time, this grid cell already has the data of the previously added point.

[0104] Assume that the distance from the point in grid cell (3,4) to the lidar is 5 meters, and the distance from the new point to the lidar is 4.8 meters. When fusion processing, the weighted average method can be used. If the weight of the new point is set to 0.6 and the weight of the old point is set to 0.4 (the weight setting can be determined according to actual conditions, such as the degree of trust in the new data). The distance after fusion is: (0.6×4.8+0.4×5)=4.88 meters.

[0105] Assume that the reflection intensity of the previous point is 0.7 and the reflection intensity of the new point is 0.8. The weighted average method is also used (assuming that the weight remains unchanged), and the fused reflection intensity is: (0.6×0.8+0.4×0.7)=0.76. Then the fused distance and reflection intensity information are updated to the grid unit (3,4).

[0106] S172, modifying the existing grid information, which includes: determining the part of the information that needs to be modified in the existing grid unit according to the newly received point cloud data and the real-time position of the vehicle; and updating the information stored in the grid unit that needs to be modified.

[0107] For example, the newly received point cloud data of the LiDAR shows that an obstacle, such as a stone, has appeared in an area that was originally thought to be open and drivable (corresponding to a grid cell). By analyzing the newly received point cloud data, it can be determined that the grid cell containing the stone needs to modify the information. For the grid cell where the stone appears, the drivable information originally stored internally (such as marked as "Drivable: Yes") needs to be modified to "Drivable: No", and some descriptive information about the obstacle (stone) can also be added, such as size, shape (assuming that it can be roughly analyzed through the point cloud data), etc.

[0108] S173, deleting invisible grid cells, which includes: determining the portion of the grid cells that is outside the visible range of the laser radar based on the current position of the vehicle and the field of view of the laser radar; and deleting the grid cells that are outside the visible range of the laser radar from the local grid map.

[0109] For example, there are some grid cells 55 meters behind the vehicle. Since they are beyond the effective detection distance of 40 meters of the LiDAR, these grid cells are outside the visible range of the LiDAR. For those grid cells that are determined to be outside the visible range of the LiDAR, such as the grid cells 55 meters behind the vehicle, they will be deleted from the local grid map. Doing so can reduce the amount of map data and improve data processing efficiency. If these invisible grid cells are not deleted, the algorithm may unnecessarily calculate these irrelevant areas during subsequent path planning. After deleting them, the computing resources can be concentrated on the visible areas related to the current driving of the vehicle.

[0110] In summary, the local grid map generated by the present application can accurately describe the environment around the vehicle in a discretized manner, and as time goes by, the local grid map can be continuously updated to reflect the dynamic changes of the environment. In addition, in the local grid map, when it is necessary to find a specific area (such as finding the parking area closest to the vehicle), the spatial index structure can quickly locate the grid cells that may contain the target area. For some complex query operations, such as querying all drivable areas within a certain range, the spatial index structure can avoid calculating those grid cells that are obviously not within the query range. Since the spatial index structure can reduce unnecessary queries and calculations, memory and computing resources are also saved accordingly. When processing large-scale local grid maps, the spatial index structure can ensure that the system runs efficiently under limited resources. In addition, the local grid map provides accurate environmental information for reliable path planning, so that a feasible path from the starting point to the end point can be searched on the local grid map. And the information in the grid map can be used to evaluate the cost of different paths, and the path with the lowest total cost can be selected based on these cost information, thereby generating a more optimized and reliable path planning. Obstacle information in the grid map can also help the path planning algorithm avoid dangerous areas.

[0111] The data processing device provided by the present application is described below. The data processing device and the data processing method described below can be referenced to each other.

[0112] Please refer to Figure 2 , Figure 2 A data processing device 200 includes a data receiving module 210 , a grid mapping module 220 and a map generating module 230 .

[0113] Exemplarily, the data receiving module 210 is used to receive point cloud data output by a laser radar, wherein each point contains three-dimensional coordinate information in a laser radar coordinate system.

[0114] Exemplarily, the grid mapping module 220 is used to perform a grid mapping operation on each point converted to the target coordinate system according to preset grid map parameters to determine the grid unit to which each point belongs.

[0115] Exemplarily, the map generation module 230 is used to obtain a local grid map based on the grid cells determined by all point cloud data points, wherein the local grid map marks the state of the vehicle's surrounding environment in a discrete form, and each grid cell marks the state of the environment within its preset range.

[0116] Please refer to Figure 3 , Figure 3 3 is a schematic diagram of the structure of the autonomous driving system provided by the present application. An autonomous driving system 300 includes a perception module 310 , a data processing module 320 and a control module 330 .

[0117] Exemplarily, the perception module 310 includes a laser radar for collecting point cloud data of the surrounding environment in real time.

[0118] Specifically, the laser radar generates high-density three-dimensional point cloud data by scanning the surrounding environment, which reflects the position, shape and motion status of objects around the vehicle. The collected point cloud data is transmitted to the point cloud processing unit through the data transmission system.

[0119] Exemplarily, the data processing module 320 is used to execute the steps of the data processing method described in any of the above embodiments.

[0120] Specifically, the data processing module 320 receives point cloud data from the laser radar and processes and analyzes the data through a local rasterization algorithm. The local rasterization algorithm converts continuous point cloud data into discrete local maps, where each grid represents the state of the environment within a certain range. The algorithm takes into account the real-time position and direction of movement of the vehicle and dynamically updates the local map to ensure the real-time and accuracy of the map.

[0121] Exemplarily, the control module 320 is used to perform path planning, obstacle detection and obstacle avoidance control based on the local grid map and real-time status information of the vehicle.

[0122] Specifically, the path planning algorithm of the control module 320 can determine the optimal path and driving strategy based on the target position of the vehicle and the current environmental conditions. The obstacle detection algorithm can analyze the obstacle information in the local map, identify potential collision risks and generate warning signals. The obstacle avoidance control algorithm can implement emergency braking or steering operations based on the detected obstacle information to ensure safe driving of the vehicle.

[0123] In order to achieve efficient point cloud data processing and real-time map updates, the local rasterization algorithm can adopt the following technologies and strategies:

[0124] First, multi-threaded parallel computing. With the parallel computing capabilities of multi-core processors or GPUs, the point cloud data processing and map update process are accelerated, thereby improving the system's response rate and real-time performance. In actual application scenarios, multi-core processors or GPUs can process multiple data subtasks at the same time, thereby effectively shortening the time cycle of point cloud data processing and map updates, ensuring that the system can respond to environmental changes in a timely manner.

[0125] Second, hardware acceleration technology. Use dedicated hardware accelerators such as FPGA (field programmable gate array) or ASIC (application-specific integrated circuit) to optimize the computing performance of the rasterization algorithm. This helps to reduce processing delays to meet real-time processing requirements in complex environments. For example, in complex urban traffic environments or large-scale industrial scenarios, hardware acceleration technology can ensure that the rasterization algorithm can still complete computing tasks quickly and accurately when faced with a large amount of point cloud data, thereby achieving real-time map updates.

[0126] Third, algorithm optimization and data structure design. Design efficient data structures for storing raster maps, such as sparse data structures or compressed storage formats. This can optimize memory utilization and reduce storage costs. At the same time, by optimizing the computational complexity and spatial complexity of the algorithm, the overall efficiency and performance of the algorithm can be comprehensively improved. When processing large-scale point cloud data to construct raster maps, a reasonable data structure can reduce memory usage, and the optimized algorithm can process data more quickly, improving the overall operating efficiency of the system.

[0127] It should be noted here that the above-mentioned data processing device provided in the embodiment of the present application and the automatic driving system can implement all the method steps implemented in the above-mentioned method embodiment and can achieve the same technical effect. The parts and beneficial effects that are the same as those in the method embodiment will not be described in detail here.

[0128] Figure 4 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 4 As shown, the electronic device may include: a processor (Processor) 410, a communication interface (Communications Interface) 420, a memory (Memory) 430 and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the data processing method.

[0129] In addition, the logic instructions in the above-mentioned memory 430 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including a number of instructions to ensure that a computer device (which can be a personal computer, a server, or a network device, etc.) executes all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.

[0130] On the other hand, the present application also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the data processing methods provided by the above methods.

[0131] On the other hand, the present application also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the data processing methods provided above.

[0132] The embodiments of the present application provide an electronic device, a computer program product, and a processor-readable storage medium, on which the computer program stored enables the processor to implement all the method steps implemented in the above-mentioned method embodiments and to achieve the same technical effects. The parts and beneficial effects of this embodiment that are the same as those in the method embodiments will not be described in detail herein.

[0133] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0134] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions to ensure that a computer device (which can be a personal computer, a server, or a network device, etc.) executes the methods described in each embodiment or some parts of the embodiments.

[0135] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Receive point cloud data output by the laser radar, where each point contains three-dimensional coordinate information in the laser radar coordinate system; According to the preset grid map parameters, a grid mapping operation is performed on each point converted to the target coordinate system to determine the grid unit to which each point belongs; Based on the grid cells determined by all the point cloud data points, a local grid map is obtained, and the local grid map marks the state of the vehicle's surrounding environment in a discrete form, and each grid cell marks the environment state within its preset range.

2. The data processing method according to claim 1, characterized in that: The method further includes dynamically updating the local grid map, which includes adding newly received point cloud data: For the newly received point cloud data, each point therein is mapped to a corresponding grid unit; If the grid cell is empty, the information of the new point is added to the grid cell; If the grid cell already has data, the new point information will be fused with the existing information.

3. The data processing method according to claim 1, characterized in that: The method also includes dynamically updating the local grid map, which includes modifying existing grid information: Determine the part of the information that needs to be modified in the existing grid cells based on the newly received point cloud data and the real-time position of the vehicle; For the grid cells whose information needs to be modified, the information stored inside is updated.

4. The data processing method according to claim 1, characterized in that: The method further includes dynamically updating the local grid map, which includes deleting invisible grid cells: According to the current position of the vehicle and the field of view of the laser radar, the portion of the grid cell that is outside the visible range of the laser radar is determined; Removes grid cells from the local grid map that are outside the visible range of the lidar.

5. The data processing method according to claim 1, characterized in that: The method further includes constructing a spatial index structure, wherein the spatial index structure is used to map the point cloud data to corresponding grid cells, and the construction steps include: A coordinate axis in the target coordinate system is selected as an initial segmentation axis, and a sorting operation is performed on all point cloud data according to the initial segmentation axis to segment them into two subsets; For the two subsets that have been segmented, a construction operation is performed recursively, including selecting the next coordinate axis of the target coordinate system as the next segmentation axis in each subset, and performing a sorting operation on the point cloud data points in the corresponding subset based on the next segmentation axis; This process is repeated in this way until the number of points contained in the subset is less than or equal to the preset threshold, and a complete spatial index structure is obtained.

6. The data processing method according to claim 1, characterized in that: The method also includes using a spatial index structure to search for points located in a specific area in the point cloud data to implement range query and nearest neighbor query, wherein the range query is to find all points in the specific area, and the nearest neighbor query is to find the point closest to a specific point.

7. The data processing method according to claim 6, characterized in that: The step of using the spatial index structure to search for points located in a specific area in the point cloud data to implement range query and nearest neighbor query includes: When performing a raster mapping operation, for a specific area corresponding to each raster cell, the range query is used to locate points that may be within the specific area; When it is necessary to determine whether a point belongs to a grid cell, and the point is close to the grid cell boundary, the nearest neighbor query is used to find the point closest to the point to determine whether the point belongs to the current grid cell.

8. The data processing method according to claim 1, characterized in that: The method further includes generating a path plan based on the local grid map, which includes: The current position of the vehicle is set as the starting point, and the destination position to be reached by the vehicle is set as the target point, and the starting point is used as the starting node to search all possible adjacent nodes of the starting node; For each adjacent node, calculate the total cost from the starting point to the target point, where the total cost is the sum of a first cost from the starting point to the adjacent node and a second cost from the adjacent node to the target point; Mark the adjacent node with the smallest total cost as the current node, continue searching for the adjacent nodes of the current node, and repeatedly perform the total cost calculation operation until the target point is reached, and connect the path nodes from the starting point to the target point to generate a path plan; In the search process, the grids marked as obstacles in the local grid map are not considered as extendable path nodes.

9. The data processing method according to claim 1, characterized in that: The method further includes assigning different values ​​to each grid cell to mark the environmental status, wherein the environmental status includes static obstacles, dynamic obstacles and no obstacles; If there is a static obstacle in the grid cell, the first value is assigned; If there is a dynamic obstacle in the grid cell, a second value is assigned; If there is no obstacle in the grid cell, a third value is assigned; The first value, the second value and the third value are different.

10. A data processing device, characterized in that: The device comprises: A data receiving module is used to receive point cloud data output by the laser radar, wherein each point contains three-dimensional coordinate information in the laser radar coordinate system; A grid mapping module is used to perform a grid mapping operation on each point converted to the target coordinate system according to preset grid map parameters to determine the grid unit to which each point belongs; The map generation module is used to obtain a local grid map based on the grid cells determined by all point cloud data points. The local grid map marks the state of the vehicle's surrounding environment in a discrete form, and each grid cell marks the environment state within its preset range.

11. An automatic driving system, characterized in that: The system comprises: A perception module, which includes a laser radar and is used to collect point cloud data of the surrounding environment in real time; A data processing module, configured to execute the steps of the data processing method according to any one of claims 1 to 9; The control module is used for path planning, obstacle detection and obstacle avoidance control based on the local grid map and the real-time status information of the vehicle.

12. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that: When the processor executes the computer program, the steps of the data processing method according to any one of claims 1 to 9 are implemented.

13. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the data processing method according to any one of claims 1 to 9 are implemented.

Images