Data processing method, device, electronic device and computer storage medium

Abstract

The embodiments of the present application provide a data processing method, device, electronic device and computer storage medium. The data processing method includes: obtaining three-dimensional point cloud data and reflection value information collected by a laser radar, wherein the three-dimensional point cloud data is used to indicate the geometric information of the target object in the environment where the laser radar is located, and the reflection value information is used to indicate the reflection value of the laser signal emitted by the laser radar at different reflection points on the target object, and the three-dimensional point cloud data includes point cloud data of ground points and point cloud data of non-ground points; according to the point cloud data of the ground points and the reference point data corresponding to the ground points, the first posture parameter in the predicted posture of the laser radar is corrected; according to the point cloud data of the non-ground points and the reference point data corresponding to the non-ground points, as well as the reflection value of the ground points and the reference reflection value corresponding to the ground points, the second posture parameter in the predicted posture is corrected. The posture correction effect of this data processing method is better.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a data processing method, device, electronic device, and computer storage medium. Background Art

[0002] The laser positioning module in the autonomous driving system can provide real-time position and posture information for the autonomous driving vehicle. The existing laser positioning module may fail in position and posture positioning in scenarios with large environmental changes. At this time, the value of the laser odometry is reflected. A robust laser odometry can continue to provide stable relative positioning attitude output when position and posture positioning fails, so that the autonomous driving vehicle can be positioned normally. Summary of the invention

[0003] In view of this, an embodiment of the present application provides a data processing solution to at least partially solve the above-mentioned problem.

[0004] According to a first aspect of an embodiment of the present application, a data processing method is provided, comprising: obtaining three-dimensional point cloud data and reflection value information collected by a laser radar, wherein the three-dimensional point cloud data is used to indicate geometric information of a target object in an environment in which the laser radar is located, and the reflection value information is used to indicate reflection values ​​of different reflection points on the target object to a laser signal emitted by the laser radar, and the three-dimensional point cloud data comprises point cloud data of ground points and point cloud data of non-ground points; correcting posture parameters other than the first posture parameter in the predicted posture of the laser radar according to the point cloud data of the ground points and the reference point data corresponding to the ground points; correcting the second posture parameter in the predicted posture according to the point cloud data of the non-ground points and the reference point data corresponding to the non-ground points, as well as the reflection value of the ground point and the reference reflection value corresponding to the ground point.

[0005] According to a second aspect of an embodiment of the present application, a data processing device is provided, comprising: an acquisition module, for acquiring three-dimensional point cloud data and reflection value information collected by a laser radar, wherein the three-dimensional point cloud data is used to indicate geometric information of a target object in an environment in which the laser radar is located, and the reflection value information is used to indicate reflection values ​​of different reflection points on the target object to a laser signal emitted by the laser radar, and the three-dimensional point cloud data includes point cloud data of ground points and point cloud data of non-ground points; a first correction module, for correcting posture parameters other than the first posture parameter in the predicted posture of the laser radar according to the point cloud data of the ground points and the reference point data corresponding to the ground points; and a second correction module, for correcting the second posture parameter in the predicted posture according to the point cloud data of the non-ground points and the reference point data corresponding to the non-ground points, as well as the reflection value of the ground point and the reference reflection value corresponding to the ground point.

[0006] According to a third aspect of an embodiment of the present application, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the data processing method described in the first aspect.

[0007] According to a fourth aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the data processing method described in the first aspect is implemented.

[0008] According to the data processing scheme provided in the embodiment of the present application, when correcting the posture parameters in the predicted posture, not only the point cloud data of the ground points and the point cloud data of the non-ground points are used, but also the reflection values ​​and the reference reflection values ​​of the ground points are used. Therefore, the posture parameters that can be properly corrected are more accurate, and accurate posture can also be obtained in scenes where the geometric information of the target object indicated by the point cloud data of the ground points and the point cloud data of the non-ground points is not obvious, but the reflection value information is relatively obvious, thereby improving the robustness of the posture prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0010] Figure 1A A flowchart of a data processing method according to Embodiment 1 of the present application;

[0011] Figure 1B for Figure 1A A schematic diagram of an example scenario in the illustrated embodiment;

[0012] Figure 2A is a flowchart of a data processing method according to Embodiment 2 of the present application;

[0013] Figure 2B A schematic diagram of a method for constructing a target map;

[0014] Figure 2C A schematic diagram for visualization of a three-dimensional map portion in a target map;

[0015] Figure 2D A schematic diagram for visualization of a two-dimensional map portion of a target map;

[0016] Figure 3 is a structural block diagram of a data processing device according to Embodiment 3 of the present application;

[0017] Figure 4 This is a schematic diagram of the structure of an electronic device according to the fourth embodiment of the present application. DETAILED DESCRIPTION

[0018] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0019] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application.

[0020] Embodiment 1

[0021] Reference Figure 1A , showing a schematic flow chart of the steps of the data processing method of the first embodiment of the present invention.

[0022] In this embodiment, the method can be applied to the laser odometer of the autonomous driving system to obtain the position and posture of the laser radar in the laser odometer relative to the environment. Of course, in other embodiments, the method can also be applied to other appropriate scenarios. The method can be executed by the laser radar, or by a data processing unit (such as a data processing chip) connected to the laser radar with data processing capabilities.

[0023] The method comprises the following steps:

[0024] Step S102: Acquire the three-dimensional point cloud data and reflection value information collected by the laser radar.

[0025] The three-dimensional point cloud data is used to indicate the geometric information of the target object in the environment where the laser radar is located. The reflection value information is used to indicate the reflection value of the laser signal emitted by the laser radar at different reflection points on the target object.

[0026] For example, the laser radar emits laser outward. When the laser is reflected by a reflection point on a target object in the environment, the laser radar can receive the laser signal reflected by the reflection point and determine the three-dimensional position data of the reflection point in the laser coordinate system of the laser radar based on the reflected laser. The three-dimensional position data includes but is not limited to the horizontal axis coordinate X, vertical axis coordinate Y and height coordinate Z of the reflection point in the laser coordinate system.

[0027] The target object can be a ground object or a non-ground object, such as a tree, a wall, a pedestrian, and a vehicle. The reflection points can include ground points and non-ground points. Since there can be multiple reflection points on a target object to reflect the laser, the three-dimensional position data corresponding to the multiple reflection points collected by the LiDAR constitutes three-dimensional point cloud data, which includes point cloud data of ground points and point cloud data of non-ground points.

[0028] In addition to obtaining three-dimensional point cloud data of reflection points, laser radar based on reflected laser can also obtain the reflection value corresponding to each reflection point. The reflection value is used to indicate the reflection intensity of the laser signal at the reflection point.

[0029] Step S104: According to the point cloud data of the ground point and the reference point data corresponding to the ground point, the posture parameters other than the first posture parameter in the predicted posture of the laser radar are corrected.

[0030] The predicted pose of the LiDAR is used to indicate the pose of the LiDAR in the reference coordinate system corresponding to the environment. Usually, the predicted pose includes the horizontal axis coordinate X, the vertical axis coordinate Y, the height coordinate Z, the yaw angle, the roll angle, and the pitch angle. The predicted pose can be determined based on the IMU device on the autonomous driving vehicle, or it can be the pose predicted based on the three-dimensional point cloud data and reflection value information collected by the LiDAR at the previous moment. Based on the predicted pose, the conversion between the laser coordinate system and the reference coordinate system can be achieved, such as mapping the three-dimensional point cloud data to the reference coordinate system of the environment, thereby determining the position of the target object in the environment.

[0031] Reference point data may be used to indicate the location of a reference point in the environment in a reference coordinate system.

[0032] In one feasible method, a target map can be constructed based on the three-dimensional point cloud data and reflection value information collected by the lidar at the current moment and the three-dimensional point cloud data and reflection value information at historical moments. The target map includes a three-dimensional map part and a two-dimensional map part. The three-dimensional map part includes reference point data, and the two-dimensional map part includes reference reflection values.

[0033] For example, the 3D map is partially determined based on the 3D point cloud data collected at the current moment and the 3D point cloud data at the historical moment. For example, the 3D point cloud data at different moments are clustered to obtain multiple clusters, and then the centroid of the cluster is calculated, and the centroid is used as the reference point. The reference point data is the position of the centroid in the reference coordinate system.

[0034] The two-dimensional map part can be obtained in the following way: rasterize the map corresponding to the environment to obtain a two-dimensional grid map, and determine the grid where the reflection point is located according to the three-dimensional position data of the reflection point corresponding to the reflection value information collected at the current moment and the three-dimensional position data of the reflection point corresponding to the reflection value information at the historical moment. The reflection value of the reflection point in each grid is calculated as the reflection value corresponding to the grid, and the reflection value of each grid is the reference reflection value.

[0035] Of course, in other embodiments, the target map can be constructed only based on the three-dimensional point cloud data and reflection value information collected at the current moment, or the target map can be constructed only through the three-dimensional point cloud data and reflection value information at historical moments, and this embodiment does not limit this.

[0036] Since there may be a deviation between the predicted pose of the lidar and the actual pose of the lidar, in order to accurately obtain the pose of the lidar at the current moment, the first pose parameter in the predicted pose is corrected through the point cloud data of the ground point in the three-dimensional point cloud data and the reference point data corresponding to the ground point.

[0037] In this embodiment, the first posture parameter is the remaining posture parameters other than the two-dimensional posture parameters in the predicted posture, such as the height coordinate, the roll angle and the pitch angle.

[0038] Step S106: Correct the second posture parameter in the predicted posture according to the point cloud data of the non-ground point and the reference point data corresponding to the non-ground point, as well as the reflection value of the ground point and the reference reflection value corresponding to the ground point.

[0039] In this embodiment, the second posture parameters can be two-dimensional posture parameters in the predicted posture, which include horizontal axis coordinates, vertical axis coordinates and yaw angles. Since the three-dimensional position data of non-ground points and the reflection values ​​of ground points have a more obvious constraint effect on the accuracy of the second posture parameters, the point cloud data of non-ground points, the reflection values ​​of ground points, reference point data corresponding to non-ground points and reference reflection values ​​can be comprehensively utilized to correct the second posture parameters in the predicted posture.

[0040] This makes it possible to use not only the three-dimensional point cloud data, that is, the geometric information of the target object, but also the reflection value information when correcting the predicted pose, thereby improving the accuracy of the correction and ensuring the robustness of the laser odometer.

[0041] The following is a description of the implementation process of this method in combination with a specific usage scenario:

[0042] like Figure 1BAs shown, in this usage scenario, it is assumed that the laser radar is installed on an autonomous driving vehicle, and the driving environment of the autonomous driving vehicle is road A, and the environment also includes target object B which is a tree and target object C which is a pedestrian.

[0043] At the kth moment, a data frame image of the kth moment collected by the laser radar is obtained, and the data frame includes three-dimensional point cloud data and reflection value information, the three-dimensional point cloud data includes three-dimensional position data of multiple ground reflection points, which form point cloud data of ground points, and three-dimensional position data of multiple non-ground reflection points, which form point cloud data of non-ground points. Similarly, the reflection value information includes reflection values ​​of ground points and reflection values ​​of non-ground points.

[0044] In addition, if the difference between the predicted pose of the lidar at the kth moment and the pose corresponding to the previous key frame meets the set conditions, the data frame at the kth moment is used as a new key frame and updated to the target map set, and the reference point data and reference reflection value corresponding to the kth moment are determined based on the key frame data in the updated target map set.

[0045] Based on the point cloud data of the ground point and the reference point data corresponding to the ground point, the first pose parameter in the predicted pose of the laser radar is corrected. The first pose parameter includes: height coordinate, roll angle and pitch angle.

[0046] Based on the corrected first pose parameters and in combination with the second pose parameters in the predicted pose, the second pose parameters are corrected according to the point cloud data of the non-ground points and the reference point data corresponding to the non-ground points, the reflection values ​​of the ground points and the reference reflection values ​​corresponding to the ground points, so that the corrected second pose parameters and the first pose parameters can be used as new predicted poses.

[0047] Since the geometric information and reflection value information indicated by the three-dimensional point cloud data are used simultaneously to match the data frame at the kth moment with the target map, the two kinds of information complement each other. In the scene where the geometric information of the target object is not obvious but the reflection value information is obvious, the predicted pose has a higher robustness. Conversely, in the scene where the reflection value information is not obvious but the geometric information of the target object is obvious, the predicted pose has a higher robustness.

[0048] Through this embodiment, when correcting the posture parameters in the predicted posture, not only the point cloud data of the ground points and the point cloud data of the non-ground points are used, but also the reflection values ​​and the reference reflection values ​​of the ground points are used. Therefore, the posture parameters that can be properly corrected are more accurate, and accurate posture can be obtained in scenes where the geometric information of the target object indicated by the point cloud data of the ground points and the point cloud data of the non-ground points is not obvious, but the reflection value information is relatively obvious, thereby improving the robustness of the posture prediction.

[0049] The data processing method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: a server, a mobile terminal (such as a mobile phone, a PAD, etc.) and a PC, etc.

[0050] Embodiment 2

[0051] Reference Figure 2A , showing a step flow chart of the data processing method of embodiment 2 of the present application.

[0052] In this embodiment, the method includes the following steps:

[0053] Step S202: Obtain three-dimensional point cloud data and reflection value information collected by the laser radar.

[0054] The three-dimensional point cloud data includes ground cloud data and point cloud data of non-ground points. Accordingly, the reflection value information includes reflection values ​​of ground points and reflection values ​​of non-ground points.

[0055] Since the method and principle of obtaining the three-dimensional point cloud data and the reflection value information are as described in the first embodiment, they will not be described in detail.

[0056] Step S202a: Obtain the target map.

[0057] The target map is constructed by fusing data from multiple key frames. In order to avoid the target map data processing volume being too large, which affects the processing speed, and to avoid the adverse effect of old data on the correction of the predicted pose, each target map has a maximum number of fused frames. Once the number of key frames in the target map set corresponding to the target map reaches the maximum number of fused frames, the target map is discarded and a new target map is used.

[0058] In order to improve the accuracy of predicted pose correction, the key frames used between adjacent target maps have a certain degree of overlap, such as Figure 2B As shown, some key frames will participate in the construction of two adjacent target maps.

[0059] Figure 2B A schematic diagram of a feasible target map construction is shown, wherein a plurality of different data frames are collected at different times, some of which are key frames, represented by solid-line frames, and other data frames are non-key frames, represented by dotted-line frames.

[0060] For data frame A, which is a key frame and a key frame that overlaps between adjacent target maps, data frame A is only added to the target map set of target map 2, and this action is indicated by a thick dotted arrow. On the other hand, data frame A is added to the target map set of target map 1 and matched with target map 1, and this action is indicated by a thick solid arrow.

[0061] For data frame B, it is a non-key frame and only needs to be matched with the currently used target map 1, and this action is represented by a thin solid line.

[0062] Combine the following Figure 2B , the construction process of the target map is described as follows:

[0063] Based on the data frame at the current moment, if the current moment is the first moment, the data frame at the current moment is the key frame (the frame is not in Figure 2B ), the key frame is updated to the target map set of target map 1. Alternatively, if the current moment is not the first moment, but the deviation of at least one of the position and posture of the predicted posture corresponding to the data frame at the current moment and the position and posture of the corrected posture corresponding to the previous key frame is greater than a set threshold (the set threshold can be determined as needed), the data frame at the current moment is used as a key frame, such as data frame A.

[0064] For keyframes, if the keyframe corresponds to the currently used target map (such as Figure 2B If the number of key frames in the target map set corresponding to the data frame A (the currently used target map is target map 1) has met the split number, the key frames are added to the target map set of the currently used target map (i.e., target map 1), and at the same time, they are added to the target map set of the next target map (i.e., target map 2).

[0065] When determining the target map based on keyframes in the target map collection:

[0066] When determining the 3D map part of the target map: cluster the 3D point cloud data in the target map set, determine the centroid of the cluster based on the clustered clusters, and use the centroid of the cluster as the reference point data. The 3D map part includes the geometric contour information of the physical environment. The schematic diagram of the visualization of the 3D map part is as follows: Figure 2C shown.

[0067] When determining the two-dimensional map portion of the target map: obtain the rasterized target map where the environment is located, and the corresponding two-dimensional grid map. According to the reflection value information, determine the reflection value of the reflection points included in each grid, calculate the average reflection value of the reflection points in the grid, and use the reflection value of each grid as the reference reflection value.

[0068] A schematic diagram of the visualization of the two-dimensional map is shown below: Figure 2D shown.

[0069] Since the data of non-key frames do not participate in updating the target map, the non-key frames only need to be matched with the target map to correct the predicted pose. The correction process can be achieved through the following steps.

[0070] Step S204: Correcting the first pose parameter in the predicted pose of the laser radar according to the point cloud data of the ground point and the reference point data corresponding to the ground point.

[0071] Since not all the reference point data included in the target map are related to the ground point, the reference point data corresponding to the ground point can be determined from all the reference point data based on the predicted pose.

[0072] For example, in one feasible manner, step S204 includes the following sub-steps:

[0073] Sub-step S2041: Determine, based on the predicted position and posture of the laser radar and the point cloud data of the ground points, a first mapping position of each ground point mapped to the reference coordinate system where the reference point data is located.

[0074] Assume that the predicted pose is represented as (x0, y0, z0, roll0, pitch0, yaw0). For the three-dimensional position data of each ground point in the point cloud data of the ground point, the three-dimensional position data and the predicted pose are calculated to determine the first mapping position of each ground point mapped to the reference coordinate system.

[0075] For example, the point cloud data of the ground points includes the three-dimensional position data of 10 ground points, and the first mapping positions of the 10 ground points mapped to the reference coordinate system are determined respectively according to the predicted postures.

[0076] Sub-step S2042: Correcting the height coordinate, roll angle and pitch angle in the predicted posture according to the reference point data corresponding to the first mapping position and the ground point.

[0077] In one feasible manner, sub-step S2042 includes the following process:

[0078] Process A1: for each of the ground points, according to a first mapping position corresponding to the ground point, determine K reference points adjacent to the first mapping position as reference points corresponding to the ground point.

[0079] Still taking the previous 10 ground points as an example, for each ground point, according to the first mapping position of the ground point, the K reference points closest to the first mapping position are determined from all reference point data of the target map as the reference points corresponding to the ground point.

[0080] Process B1: Determine the ground reference plane corresponding to each ground point based on the data of the reference point corresponding to each ground point.

[0081] For each ground point, the reference point corresponding to the ground point forms a ground reference plane, that is, the ground reference plane is determined based on the data of the reference point corresponding to the ground point.

[0082] The method for determining the ground reference plane is, for example, to randomly select an initial point from K reference points, select the point closest to the initial point from the remaining K-1 reference points to form a connecting line, and then select the point closest to the connecting line from the remaining K-2 reference points, connect the point to the two ends of the connecting line respectively, and then use the two connected lines as new connecting lines, repeat the process of selecting the closest point from the remaining reference points and connecting the closest point to the two ends of the connecting line until all the reference points are connected together, and determine the ground reference plane based on the connected reference points.

[0083] Process C1: Correcting the height coordinate, roll angle and pitch angle according to the ground reference distance between each ground point and the corresponding ground reference plane.

[0084] For each ground point, the ground reference distance between each ground point and the corresponding ground reference plane can be calculated. The ground reference distance can be expressed as: Among them, i represents the i-th ground point, and point-to-plane represents the distance from the ground point to the corresponding ground reference plane.

[0085] The ground target difference is determined based on the sum of the squares of the ground reference distances of multiple ground points. The ground target difference can be expressed as:

[0086] Based on the ground target difference, the predicted pose can be corrected according to the Gauss-Newton iteration method to obtain a candidate corrected pose. The candidate corrected pose can be expressed as (x1, y1, z1, roll1, pitch1, yaw1).

[0087] In order to ensure the accuracy of the correction, only some parameters in the candidate correction pose (that is, the first pose parameters), namely (z1, roll1, pitch1) are selected as the valid part, where z is the height coordinate, roll is the roll angle, and pitch is the roll angle.

[0088] The corrected first pose parameters and the original two-dimensional pose parameters in the predicted pose (i.e., the second pose parameters) are combined into a transition pose. The transition pose can be expressed as (x0, y0, z1, roll1, pitch1, yaw0), so that the second pose parameters can be corrected according to the transition pose parameters in subsequent steps.

[0089] Step S206: Correct the second posture parameter in the predicted posture according to the point cloud data of the non-ground point and the reference point data corresponding to the non-ground point, as well as the reflection value of the ground point and the reference reflection value corresponding to the ground point.

[0090] In one feasible manner, step S206 includes the following sub-steps:

[0091] Sub-step S2061: Determine the non-ground reference surface corresponding to each of the non-ground points based on the corrected first pose parameter in the predicted pose, the transition pose formed by the second pose parameter, and the reference point data corresponding to the non-ground point.

[0092] Sub-step S2061 includes the following process:

[0093] Process A2: Determine, based on the transition posture and the reference point data corresponding to the non-ground point, a second mapping position of each non-ground point mapped to the reference coordinate system where the reference point data is located.

[0094] The corrected first pose parameters and the original second pose parameters in the predicted pose constitute the aforementioned transition pose (x0, y0, z1, roll1, pitch1, yaw0).

[0095] For the multiple non-ground points included in the point cloud data of the non-ground points, the second mapping position of each non-ground point mapped to the reference coordinate system is determined according to the three-dimensional position data and transition posture of each non-ground point. For example, if the point cloud data of the non-ground points includes 20 non-ground points, the second mapping position of each non-ground point is determined respectively.

[0096] Process B2: determining, according to each of the second mapping positions, a non-ground reference surface formed by the reference point data corresponding to each of the non-ground points.

[0097] Based on the second mapping position of each non-ground point, the K reference points closest to the second mapping position can be determined from the reference point data of all reference points included in the target map, and the reference point data of these K nearest reference points can be obtained as the reference point data corresponding to the non-ground point.

[0098] For each non-ground point, the corresponding non-ground reference surface is determined using the reference point data corresponding to the non-ground point. The process of determining the non-ground reference surface is similar to the process of determining the ground reference surface in the aforementioned process B1, so it will not be repeated here.

[0099] Sub-step S2062: determining the non-ground distance between each non-ground point and the corresponding non-ground reference surface according to the point cloud data of the non-ground point and the non-ground reference surface corresponding to each non-ground point.

[0100] The point cloud data of non-ground points includes three-dimensional position data of multiple non-ground points. For each non-ground point, the second mapping position of the non-ground point in the reference coordinate system is determined according to its three-dimensional position data and transition posture, and the distance between the second mapping position and the non-ground reference surface is calculated as the non-ground distance.

[0101] The non-ground distance can be expressed as Where j represents the jth non-ground point. Point-to-plane represents the distance from a point to a plane.

[0102] Sub-step S2063: Determine the reflection value residual between the reflection value of the ground point and the reference reflection value according to the transition posture, the reflection value of the ground point and the reference reflection value.

[0103] The transition pose is composed of the corrected first pose parameters and the original second pose parameters in the predicted pose, which can be expressed as (x0, y0, z1, roll1, pitch1, yaw0).

[0104] For each ground point, the ground point can be mapped to the two-dimensional grid map according to the three-dimensional position data and transition posture of the ground point, and the third mapping position of the ground point in the two-dimensional grid map is determined.

[0105] Determine the reference reflection value corresponding to the ground point according to the third mapping position, and calculate the reflection value of the ground point and its corresponding reference reflection value to determine the reflection value residual. The reflection value residual can be expressed as

[0106] Its calculation method can be expressed as

[0107] Among them, I m Represents a two-dimensional raster map containing reference reflectance values. ξ Represents the angle parameter matrix in the transition pose, namely the matrix of roll1, pitch1 and yaw0. ξ I represents the displacement parameter matrix in the transition pose, i.e., the matrix of x0, y0, and z1. m (π(R ξ ·p i +t ξ )) represents mapping the i-th ground point to the reference reflection value of the corresponding grid in the two-dimensional grid map. I(p i ) represents the reflection value of the i-th ground point.

[0108] Sub-step S2064: Correcting the horizontal axis coordinate, the vertical axis coordinate and the yaw angle according to the non-ground distance and the reflection value residual.

[0109] In one feasible method, the square sum of non-ground distances of multiple non-ground points and the square sum of reflection residual values ​​of multiple ground points are calculated respectively. The square sum of non-ground distances is expressed as The sum of squares of the reflection residual values ​​is expressed as

[0110] According to the preset ground point residual weight w g and the non-ground point residual weights w o , determine the non-ground target difference. The calculation of the non-ground target difference can be expressed as:

[0111] Based on the non-ground target difference, the transition pose is corrected according to the Gauss-Newton method, and the corrected transition pose can be expressed as (x2, y2, z2, roll2, pitch2, yaw2).

[0112] The second pose parameters (x2, y2, yaw2) are selected from the corrected transition pose as valid pose parameters.

[0113] The corrected first pose parameter and second pose parameter constitute the correction result of the current correction, which is expressed as (x2, y2, z1, roll1, pitch1, yaw2). This correction result can be used as the predicted pose for the next correction.

[0114] In order to ensure the accuracy of the correction, the method may further include step S208.

[0115] Step S208: Use the corrected second pose parameter and the first pose parameter to update the predicted pose of the laser radar, and return to the action of step S204 to continue execution until the termination condition is met.

[0116] The updated predicted position and posture is the aforementioned (x2, y2, z1, roll1, pitch1, yaw2). The process can then return to step S204 and continue to execute until the termination condition is met.

[0117] The termination condition is, for example, that the difference between two adjacent corrected postures is less than a set value (the set value can be determined as needed). Alternatively, the termination condition can be other appropriate conditions, such as convergence of the corrected posture.

[0118] Through the above process, a two-step iterative optimization method is adopted to optimize (z, roll, pitch) and (x, y, yaw) respectively. Among them, the optimization of (x, y, yaw) simultaneously uses the reflection value information and the geometric information carried in the three-dimensional point cloud data, and puts these two kinds of information into an optimization framework for joint optimization.

[0119] Specifically, (z, roll, pitch) is optimized by matching the point cloud data of ground points with the reference point data in the target map, and (x, y, yaw) is optimized by using the point cloud data of non-ground points and the reference point data in the target map, as well as the reflection values ​​of ground points and the reference reflection values ​​included in the two-dimensional grid map.

[0120] This two-step iterative optimization method puts the geometric information and reflection value information of the 3D point cloud data collected by the LiDAR into an optimization framework for joint optimization to obtain robust and accurate object poses, improving robustness and accuracy. This not only obtains an optimized 6DOF pose, but also makes full use of the reflection value information to improve robustness.

[0121] This method can be applied in autonomous driving scenarios to accurately estimate the vehicle's position and posture. This method can be applied to different types of autonomous driving vehicles, for example, logistics vehicles, public service vehicles, medical service vehicles, terminal service vehicles, etc. As long as the vehicle is equipped with a lidar, this method can be used for position and posture estimation.

[0122] For logistics vehicles, they can transport goods to appropriate locations according to set routes during autonomous driving. During autonomous driving, the position and posture of the physical vehicle are determined based on the 3D point cloud map and 2D reflectance value grid map collected by the lidar.

[0123] Similarly, for public service vehicles, such as fire trucks, de-icing trucks, sprinkler trucks, snowplows, garbage disposal vehicles, traffic control vehicles, etc., these public service vehicles need to accurately determine their own posture in order to accurately determine their current position during the automatic driving process. For this purpose, the 3D point cloud map and 2D reflectance value grid map collected by the laser radar carried by the vehicle can be used to determine the posture of the physical vehicle. The process of other vehicles with different functions is similar, so it will not be repeated.

[0124] Through this embodiment, when correcting the posture parameters in the predicted posture, not only the point cloud data of the ground points and the point cloud data of the non-ground points are used, but also the reflection values ​​and the reference reflection values ​​of the ground points are used. Therefore, the posture parameters that can be properly corrected are more accurate, and accurate posture can be obtained in scenes where the geometric information of the target object indicated by the point cloud data of the ground points and the point cloud data of the non-ground points is not obvious, but the reflection value information is relatively obvious, thereby improving the robustness of the posture prediction.

[0125] The data processing method of this embodiment can be executed by any appropriate electronic device with data processing capabilities, including but not limited to: a server, a mobile terminal (such as a mobile phone, a PAD, etc.) and a PC, etc.

[0126] Embodiment 3

[0127] Reference Figure 3 , shows a structural block diagram of a data processing device according to Embodiment 3 of the present application.

[0128] The data processing device includes:

[0129] An acquisition module 302 is used to acquire three-dimensional point cloud data and reflection value information collected by the laser radar, wherein the three-dimensional point cloud data is used to indicate geometric information of a target object in an environment where the laser radar is located, and the reflection value information is used to indicate reflection values ​​of different reflection points on the target object to a laser signal emitted by the laser radar, and the three-dimensional point cloud data includes point cloud data of ground points and point cloud data of non-ground points;

[0130] A first correction module 304 is used to correct the first and second pose parameters of the predicted pose of the laser radar according to the point cloud data of the ground point and the reference point data corresponding to the ground point;

[0131] The second correction module 306 is used to correct the second posture parameter in the predicted posture according to the point cloud data of the non-ground point and the reference point data corresponding to the non-ground point, the reflection value of the ground point and the reference reflection value corresponding to the ground point.

[0132] Optionally, the first posture parameter in the predicted posture includes: height coordinate, roll angle and pitch angle;

[0133] The first correction module 304 includes:

[0134] A first determination module 3041 is used to determine a first mapping position of each ground point mapped to a reference coordinate system where the reference point data is located according to the predicted position and posture of the laser radar and the point cloud data of the ground point;

[0135] The remaining parameter correction module 3042 is used to correct the height coordinate, roll angle and pitch angle in the predicted posture according to the reference point data corresponding to the first mapping position and the ground point.

[0136] Optionally, the remaining parameter correction module 3042 is used to determine, for each ground point, according to the first mapping position corresponding to the ground point, K reference points adjacent to the first mapping position as reference points corresponding to the ground point; determine the ground reference plane corresponding to each ground point according to the data of the reference point corresponding to each ground point; and correct the height coordinate, roll angle and pitch angle according to the ground reference distance between each ground point and the corresponding ground reference plane.

[0137] Optionally, the second posture parameter includes: a horizontal axis coordinate, a vertical axis coordinate and a yaw angle;

[0138] The second correction module 306 includes:

[0139] The second determination module 3061 is used to determine the non-ground reference surface corresponding to each non-ground point according to the corrected first posture parameter in the predicted posture, the transition posture formed by the second posture parameter and the reference point data corresponding to the non-ground point;

[0140] A third determining module 3062 is used to determine the non-ground distance between each non-ground point and the corresponding non-ground reference surface according to the point cloud data of the non-ground point and the non-ground reference surface corresponding to each non-ground point;

[0141] A fourth determination module 3063 is used to determine a reflection value residual between the reflection value of the ground point and the reference reflection value according to the transition posture, the reflection value of the ground point and the reference reflection value;

[0142] The two-dimensional parameter correction module 3064 is used to correct the horizontal axis coordinate, the vertical axis coordinate and the yaw angle according to the non-ground distance and the reflection value residual.

[0143] Optionally, the second determination module 3061 is used to determine the second mapping position of each non-ground point mapped to the reference coordinate system where the reference point data is located based on the transition posture and the reference point data corresponding to the non-ground point; and determine the non-ground reference surface formed by the reference point data corresponding to each non-ground point based on each second mapping position.

[0144] Optionally, the fourth determination module 3063 is used to determine the two-dimensional position of the ground point mapped to the two-dimensional grid map corresponding to the environment based on the transition posture and the point cloud data of the ground point; determine the reference reflection value corresponding to the ground point based on the two-dimensional position; and determine the reflection value residual based on the reflection value of the ground point and the corresponding reference reflection value.

[0145] Optionally, the device further comprises:

[0146] The loop processing module 308 is used to update the predicted pose of the laser radar using the corrected second pose parameter and the first pose parameter, and return to the first correction module 304 to continue execution until the termination condition is met.

[0147] The data processing device of this embodiment is used to implement the corresponding data processing methods in the aforementioned multiple method embodiments, and has the beneficial effects of the corresponding method embodiments, which will not be described in detail here. In addition, the functional implementation of each module in the data processing device of this embodiment can refer to the description of the corresponding parts in the aforementioned method embodiments, which will not be described in detail here.

[0148] Embodiment 4

[0149] Reference Figure 4, shows a schematic diagram of the structure of an electronic device according to the fourth embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the electronic device.

[0150] like Figure 4 As shown, the electronic device may include: a processor (processor) 402 , a communication interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0151] in:

[0152] The processor 402 , the communication interface 404 , and the memory 406 communicate with each other via a communication bus 408 .

[0153] The communication interface 404 is used to communicate with other electronic devices or servers.

[0154] The processor 402 is used to execute the program 410, and specifically can execute the relevant steps in the above data processing method embodiment.

[0155] Specifically, the program 410 may include program codes, which include computer operation instructions.

[0156] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0157] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0158] The program 410 may be specifically used to enable the processor 402 to execute the steps of the aforementioned data processing method.

[0159] The specific implementation of each step in program 410 can refer to the corresponding description of the corresponding steps and units in the above data processing method embodiment, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0160] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0161] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CDROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a special-purpose processor or programmable or special-purpose hardware (such as an ASIC or FPGA). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., RAM, ROM, flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the data processing method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the data processing method shown here, the execution of the code converts the general-purpose computer into a special-purpose computer for executing the data processing method shown here.

[0162] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0163] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. A data processing method, comprising: Acquire three-dimensional point cloud data and reflection value information collected by the laser radar, wherein the three-dimensional point cloud data is used to indicate geometric information of a target object in an environment where the laser radar is located, and the reflection value information is used to indicate reflection values ​​of different reflection points on the target object to a laser signal emitted by the laser radar, and the three-dimensional point cloud data includes point cloud data of ground points and point cloud data of non-ground points; Correcting a first pose parameter in the predicted pose of the laser radar according to the point cloud data of the ground point and the reference point data corresponding to the ground point; Correcting the second pose parameters in the predicted pose according to the corrected first pose parameters in the predicted pose, the point cloud data of the non-ground point and the reference point data corresponding to the non-ground point, and the reflection value of the ground point and the reference reflection value corresponding to the ground point; The method also includes: if the difference between the predicted pose of the laser radar and the pose corresponding to the previous key frame meets the set conditions, updating the target map set based on the three-dimensional point cloud data and the reflection value information, and determining the reference point data and the reference reflection value based on the updated target map set.

2. The method according to claim 1, wherein: The first posture parameter in the predicted posture includes: height coordinate, roll angle and pitch angle; The step of correcting the first pose parameter in the predicted pose of the laser radar according to the point cloud data of the ground point and the reference point data corresponding to the ground point comprises: Determine, according to the predicted position and posture of the laser radar and the point cloud data of the ground point, a first mapping position of each ground point mapped to the reference coordinate system where the reference point data is located; The height coordinate, roll angle and pitch angle in the predicted posture are corrected according to the reference point data corresponding to the first mapping position and the ground point.

3. The method according to claim 2, wherein: The step of correcting the height coordinate, the roll angle, and the pitch angle in the predicted posture according to the reference point data corresponding to the first mapping position and the ground point includes: For each of the ground points, according to the first mapping position corresponding to the ground point, determine K reference points adjacent to the first mapping position as reference points corresponding to the ground point; Determine the ground reference surface corresponding to each ground point according to the data of the reference point corresponding to each ground point; The height coordinate, roll angle and pitch angle are corrected according to the ground reference distance between each ground point and the corresponding ground reference plane.

4. The method according to claim 1, wherein: The second posture parameters include: a horizontal axis coordinate, a vertical axis coordinate and a yaw angle; The method of correcting the second posture parameter in the predicted posture according to the corrected first posture parameter in the predicted posture, the point cloud data of the non-ground point and the reference point data corresponding to the non-ground point, and the reflection value of the ground point and the reference reflection value corresponding to the ground point comprises: Determine the non-ground reference surface corresponding to each of the non-ground points according to the transitional posture formed by the corrected first posture parameter and the second posture parameter in the predicted posture and the reference point data corresponding to the non-ground point; Determine the non-ground distance between each non-ground point and the corresponding non-ground reference surface according to the point cloud data of the non-ground point and the non-ground reference surface corresponding to each non-ground point; Determine a reflection value residual between the reflection value of the ground point and the reference reflection value according to the transition posture, the reflection value of the ground point and the reference reflection value; According to the non-ground distance and the reflection value residual, the horizontal axis coordinate, the vertical axis coordinate and the yaw angle in the second posture parameter are corrected.

5. The method according to claim 4, wherein: The step of determining the non-ground reference surface corresponding to each non-ground point according to the transitional posture formed by the corrected first posture parameter and the second posture parameter in the predicted posture and the reference point data corresponding to the non-ground point comprises: Determine, according to the transition posture and the reference point data corresponding to the non-ground point, a second mapping position of each non-ground point mapped to the reference coordinate system where the reference point data is located; A non-ground reference surface formed by the reference point data corresponding to each of the non-ground points is determined according to each of the second mapping positions.

6. The method according to claim 4, wherein: The determining, according to the transition posture, the reflection value of the ground point and the reference reflection value, a reflection value residual between the reflection value of the ground point and the reference reflection value comprises: Determine, according to the transition posture and the point cloud data of the ground point, a two-dimensional position of the ground point mapped to a two-dimensional grid map corresponding to the environment; Determine a reference reflection value corresponding to the ground point according to the two-dimensional position; The reflection value residual is determined according to the reflection value of the ground point and the corresponding reference reflection value.

7. The method according to claim 1, wherein: The method further comprises: The predicted pose of the laser radar is updated using the corrected first pose parameter and second pose parameter, and the action of correcting the first pose parameter in the predicted pose of the laser radar is continued according to the point cloud data of the ground point and the reference point data corresponding to the ground point until the termination condition is met.

8. A data processing device, comprising: An acquisition module, used to acquire three-dimensional point cloud data and reflection value information collected by the laser radar, wherein the three-dimensional point cloud data is used to indicate geometric information of a target object in an environment where the laser radar is located, and the reflection value information is used to indicate reflection values ​​of different reflection points on the target object to a laser signal emitted by the laser radar, and the three-dimensional point cloud data includes point cloud data of ground points and point cloud data of non-ground points; A first correction module, used for correcting the posture parameters other than the first posture parameter in the predicted posture of the laser radar according to the point cloud data of the ground point and the reference point data corresponding to the ground point; A second correction module is used to correct the second posture parameters in the predicted posture according to the corrected first posture parameters in the predicted posture, the point cloud data of the non-ground point and the reference point data corresponding to the non-ground point, and the reflection value of the ground point and the reference reflection value corresponding to the ground point; Among them, the acquisition module is also used to: if the difference between the predicted posture of the laser radar and the posture corresponding to the previous key frame meets the set conditions, then update the target map set based on the three-dimensional point cloud data and the reflection value information, and determine the reference point data and the reference reflection value based on the updated target map set.

9. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the data processing method according to any one of claims 1 to 7.

10. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the data processing method according to any one of claims 1 to 7 is implemented.

Images