A GPU-accelerated collaborative simulation method for unmanned driving multi-lidar

Abstract

The present invention discloses an unmanned driving multi-lidar collaborative simulation method based on GPU acceleration, which has the following characteristics and comprises the following steps: step 1, obtaining the depth information of the laser radar at the initial position of the vehicle, and converting it into the point cloud output of the corresponding laser radar, so as to obtain the depth information and point cloud of the laser radar at the initial position; step 2, based on the latest position of the vehicle, using the GPU to generate a depth map surrounding the vehicle; step 3, constructing an optimization loss function and an optimization problem based on the depth map; step 4, simplifying the optimization loss function; step 5, finding the optimal solution of the optimization problem; step 6, judging whether the optimization loss function reaches the optimization threshold according to the optimal solution; step 7, converting the depth information of the laser radar into the corresponding point cloud.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving, and in particular to an unmanned driving multi-laser radar collaborative simulation method based on GPU acceleration. Background Art

[0002] As LiDAR is widely used in autonomous driving and used to generate point clouds, its simulation has also received more attention and challenges. Multi-line LiDAR needs to emit a large number of laser beams at high frequency, so simulation requires real-time and high-performance computing. In particular, in real simulation needs, it is often necessary to equip a single vehicle with multiple LiDARs for co-simulation, which further increases the requirements for computing resources.

[0003] However, the current unmanned driving simulation software has low simulation efficiency for lidar. For example, well-known simulation platforms such as CARLA use a CPU-based method to calculate the intersection of lasers and objects to generate point clouds. Its low efficiency is far from meeting the simulation needs. Even though the simulation platform SVLSimulator developed by LG Silicon Valley Lab uses GPU acceleration methods, it is difficult to perform efficient collaborative simulation of multiple lidars under resource constraints. Summary of the invention

[0004] The present invention is made to solve the above-mentioned problem, and its purpose is to provide an unmanned driving multi-lidar collaborative simulation method based on GPU acceleration.

[0005] The present invention provides an unmanned driving multi-lidar collaborative simulation method based on GPU acceleration, which has the following characteristics and includes the following steps: Step 1, obtaining the depth information of the laser radar at the initial position of the vehicle, and converting it into a point cloud output of the corresponding laser radar, so as to obtain the depth information and point cloud of the laser radar at the initial position; Step 2, based on the latest position of the vehicle, using the GPU to generate a depth map surrounding the vehicle; Step 3, constructing an optimization loss function and an optimization problem based on the depth map; Step 4, simplifying the optimization loss function; Step 5, finding the optimal solution to the optimization problem; Step 6, judging whether the optimization loss function reaches the optimization threshold according to the optimal solution; Step 7, converting the depth information of the laser radar into the corresponding point cloud.

[0006] The unmanned driving multi-lidar collaborative simulation method based on GPU acceleration provided by the present invention may also have the following features: wherein, in step 1, in the first frame of the simulation process, a CPU method is used to simulate each laser beam of each lidar through ray intersection of a physical engine, and the depth information relative to the lidar sensed by the laser beam at the initial position of the vehicle is obtained, and converted into a point cloud output of the corresponding lidar, and the azimuth and angle of each laser beam are recorded.

[0007] The unmanned driving multi-lidar collaborative simulation method based on GPU acceleration provided by the present invention may also have the following features: wherein, in step 2, in the latest frame of the simulation process, according to the latest position of the vehicle, a set of high-resolution depth maps surrounding the vehicle is generated using the GPU with the center of the vehicle as the viewpoint.

[0008] The unmanned driving multi-lidar collaborative simulation method based on GPU acceleration provided by the present invention may also have the following features: wherein, in step 3, based on the depth corresponding to the previous frame of the laser beam, the depth difference between the two frames of lidar is used as the optimization variable, and the depth information of the depth map of the latest frame is combined to construct an optimization loss function, which is constructed into a constrained optimization problem, specifically including the following steps: step 3-1, analyzing the assumptions and rationality of the optimization problem; step 3-2, setting the optimization variables of the optimization problem; step 3-3, optimizing the target loss function.

[0009] The unmanned driving multi-lidar collaborative simulation method based on GPU acceleration provided by the present invention may also have the following features: wherein, in step 4, a local linear interpolation method is used to interpolate the depth map to simplify the optimization loss function. Specifically, the following steps are included: step 4-1, optimizing the feasible domain of variables; step 4-2, using a primary interpolation method to simplify the optimization loss function.

[0010] The unmanned driving multi-lidar collaborative simulation method based on GPU acceleration provided by the present invention may also have the following features: wherein, in step 5, a first-order Taylor expansion is performed on the high-order terms of the optimization loss function to further simplify the loss function, thereby obtaining the analytical form of the optimal solution to the optimization problem.

[0011] The unmanned driving multi-lidar collaborative simulation method based on GPU acceleration provided by the present invention may also have the following features: wherein, in step 6, it is detected whether the optimization loss function reaches the optimization threshold to determine whether the optimization is successful. If successful, the depth of the latest frame is the sum of the depth of the previous frame and the value of the corresponding optimization variable when the loss function takes the minimum value. Otherwise, the CPU method is adopted to use the physical engine ray intersection to obtain the depth of the latest frame.

[0012] Functions and Effects of the Invention

[0013] According to the GPU-accelerated unmanned driving multi-lidar collaborative simulation method involved in the present invention, the specific steps are: Step 1, obtain the depth information of the lidar at the initial position of the vehicle, and convert it into a point cloud output of the corresponding lidar, so as to obtain the depth information and point cloud of the lidar at the initial position; Step 2, based on the latest position of the vehicle, use the GPU to generate a depth map surrounding the vehicle; Step 3, construct an optimization loss function and an optimization problem based on the depth map; Step 4, simplify the optimization loss function; Step 5, find the optimal solution to the optimization problem; Step 6, determine whether the optimization loss function reaches the optimization threshold according to the optimal solution; Step 7, convert the depth information of the lidar into the corresponding point cloud.

[0014] Therefore, the multi-lidar collaborative simulation method based on GPU acceleration proposed in the present invention is based on the characteristics of the close distance and small parallax of the laser radars in the scenario where a single vehicle is equipped with multiple laser radars, and makes full use of the point cloud information of the previous frame of simulation, and adopts GPU and numerical optimization iterative solution method to efficiently simulate multiple laser radars.

[0015] The multi-lidar collaborative simulation method based on GPU acceleration proposed in the present invention efficiently utilizes GPU and CPU computing resources, especially for the simulation task of a single vehicle equipped with multiple lidar scenarios. It can perform efficient collaborative simulation of multiple lidars, and can generate high-quality point clouds while ensuring high-efficiency simulation, ensuring that the point clouds will not be distorted. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flowchart of a multi-lidar collaborative simulation method based on GPU acceleration in an embodiment of the present invention;

[0017] Figure 2 is a schematic diagram of depth map generation in an embodiment of the present invention;

[0018] Figure 3 is a schematic diagram of an optimization loss function in an embodiment of the present invention;

[0019] Figure 4 is a schematic diagram of a primary interpolation method in an embodiment of the present invention;

[0020] Figure 5 is a point cloud image of a crossroad in an embodiment of the present invention;

[0021] Figure 6 It is a point cloud image of a tunnel scene in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following embodiments and the accompanying drawings specifically illustrate a multi-lidar collaborative simulation method based on GPU acceleration of the present invention.

[0023] In this embodiment, a multi-lidar collaborative simulation method based on GPU acceleration is provided.

[0024] First, the coordinate system used in this embodiment is explained: This embodiment uses a left-handed coordinate system for symbol representation, the X-axis represents the front-to-back direction and the positive direction is forward, the Y-axis represents the left-to-right direction and the positive direction is right, and the Z-axis represents the up-down direction and the positive direction is downward. The Euler angles (pitch, yaw, roll) are used to represent the direction of the vector. The pitch angle pitch represents the angle in the vertical direction, that is, the rotation around the Y axis, and the positive direction is upward; the yaw angle yaw represents the angle in the horizontal direction, that is, the rotation around the X axis, and the positive direction is left; the roll angle roll represents the rotation around the Z axis, and the positive direction is clockwise.

[0025] In addition, for the convenience of explanation, assume that a vehicle is equipped with three laser radars placed at different positions and angles, and the coordinate system with the center of the vehicle as the origin is denoted by Σ Car , and only consider the laser radars and record them as LiDAR1, LiDAR2, LiDAR3. And only consider one of the laser radars and record it as LiDAR * , because there is displacement and angle offset between the laser radar and the vehicle center, Σ LiDAR* and Σ Car are different.

[0026] Figure 1 It is a flowchart of a multi-lidar collaborative simulation method based on GPU acceleration in an embodiment of the present invention.

[0027] like Figure 1 As shown, the multi-lidar collaborative simulation method based on GPU acceleration involved in this embodiment includes the following steps:

[0028] Step S1, obtain the depth information of the laser radar at the initial position of the vehicle, and convert it into the point cloud output of the corresponding laser radar to obtain the depth information and point cloud of the laser radar at the initial position.

[0029] In the first frame of the simulation process, the CPU method is used to simulate each laser beam of each lidar through the physical engine ray intersection, obtain the depth information relative to the lidar sensed by the laser beam at the initial position of the vehicle, and convert it into the point cloud output of the corresponding lidar, and record the azimuth and angle of each laser beam.

[0030] The azimuth and angle of each laser beam are calculated based on the rotation speed, number of channels and other parameters of the laser radar, and the ray intersection method of the physical engine is used. * For example, Σ LiDAR* The origin of the coordinate system is taken as the starting point, and a ray is emitted in the direction of the laser beam to obtain the intersection point and the distance Σ to the intersection point. LiDAR* The distance information of the origin.

[0031] Step S2, based on the latest position of the vehicle, using the GPU to generate a depth map surrounding the vehicle.

[0032] In the latest frame of the simulation process, based on the latest position of the vehicle, a set of high-resolution depth maps surrounding the vehicle are generated using the GPU with the center of the vehicle as the viewpoint.

[0033] Figure 2 Schematic diagram of depth map generation in this embodiment.

[0034] In this embodiment, three depth maps are generated around the vehicle, with Σ Car As the reference system, with the vehicle as the viewpoint, the rotation transformation is performed around the X-axis at angles of 0, 2π / 3, and 4π / 3 respectively, to generate a depth map with a horizontal field of view of 2π / 3. The schematic diagram of the generated depth map is shown in Figure 2. In the figure, the blue plane represents the depth map, the green one is the unmanned vehicle, and the purple cylinder on the top of the vehicle represents the laser radar. The three depth maps are hereinafter referred to as map1, map2, and map3.

[0035] Step S3, constructing an optimization loss function and an optimization problem based on the depth map.

[0036] Based on the depth corresponding to the previous frame of the laser beam, the depth difference between the two frames of the laser radar is used as the optimization variable, and the depth information of the depth map of the latest frame is combined to construct an optimization loss function, which is constructed into a constrained optimization problem. The representation and solution of the optimization problem are both based on Σ Car Assume that LiDAR * The relative position is (x0, y0, z0), and the unit direction vector is (e x ,e y ,e z )'s laser.

[0037] The specific steps include:

[0038] Step S3-1, analyzing the assumptions and rationality of the optimization problem.

[0039] The optimization problem proposed in this embodiment is based on the basic assumption that the depth of the laser beam laser does not change much between two adjacent frames. This is because the time interval between the two frames is short, and the point cloud images are usually very similar between the two frames. However, in some cases, the depth of some lasers will suddenly change between two frames. For example, in a scene where the laser passes over the edge of an object, the laser in the previous frame still detects the depth of the object surface, and the depth of the laser detection in the next frame leaves the object and suddenly changes. For this situation, the assumption is not valid, and the numerical optimization algorithm cannot find the correct solution. Fortunately, the proportion of laser beams with such sudden depth changes is very small. In step S6, the loss function threshold is analyzed to determine whether the depth has suddenly changed. If a sudden change occurs, the CPU-based method is used to directly find the intersection point.

[0040] In summary, it is reasonable to perform numerical optimization based on this assumption. Based on this assumption, this algorithm uses the point cloud depth information of the previous frame and the depth map of the latest frame, combined with the numerical optimization method, to fully utilize the historical information and the latest depth information to complete the depth solution.

[0041] Step S3-2, setting the optimization variables of the optimization problem.

[0042] Let F represent the latest frame, and F-1 represent the previous frame. This optimization problem is to find the depth value of the laser in the previous frame. F-1 Under the condition of optimizing the solution of the latest frame depth F The value of depth. F and depth F-1 Assuming that the difference is not too big, we can set depth F ∈[depth F-1 -Δd,depth F-1 +Δd] where Δd is a relative F-1 A smaller value can take depth F-1 / N, where N is a constant and in this embodiment is 10, which means that the depth change is less than or equal to 1 / 10.

[0043] To do this, you can define depth F =depth F-1 +t,t∈[-depth F-1 / N,depth F-1 / N] where t is the optimization variable and depth F-1 and N can be considered as known constants.

[0044] Step S3-3, optimizing the target loss function.

[0045] Figure 3 Schematic diagram of the optimization loss function in an embodiment of the present invention.

[0046] Set up LiDAR * The displacement relative to the center of the vehicle is (x0, y0, z0), and the direction vector corresponding to the laser is (e x ,e y ,e z ), the depth corresponding to the laser in the latest frame F is depth F , which corresponds to a point P in space with coordinates (x0,y0,z0)+depth F (e x ,e y ,e z ). The center of the vehicle O is at the origin of the coordinate system, so the length of the line segment OP is ||OP||2=||x0,y0,z0)+dep F (e x ,e y ,e z )||2. Let the coordinates of the intersection of line segment OP and the latest frame rate depth map be (x map ,y map ), the depth of the intersection in the depth map is depth map =depth(x map ,y map ), where depth(x,y) is the depth value obtained by rendering the depth map at coordinates (x,y).

[0047] depth(x map ,y map ) and ||OP||2 are both increasing with depth F The value of changes continuously, while depth F =depth F-1 +t, so depth(x map ,y map ) and ||OP||2 are both continuous functions of t. The schematic diagram of the loss function is shown in Figure 3, where the dark line segment represents the laser emitted by the laser radar directly behind, and its length is the variable depth F =depth F-1 +t; the intersection of the two line segments on the far left represents the point P corresponding to the laser; the rightmost point of the light-colored line segment is the center of the vehicle O located at the origin of the coordinate system; the light-colored line segment represents OP, and the point in the middle of the light-colored line segment is the intersection point Q (x map ,y map ). It can be seen that as t changes, the position of point P also changes, and the length of line segment OP and the coordinates of the intersection point Q(x map ,y map ) also changes accordingly, and Q(x map ,ymap ) The depth corresponding to the depth map also changes accordingly. If the blue point P is the intersection of the laser and the object in the scene, then the length of the yellow line segment OP ||OP||2 is the intersection of the latest frame depth map (x map ,y map ) corresponds to the depth depth(x map ,y map ) should be equal.

[0048] Therefore, the loss function is set to the line segment length ||OP||2 and the depth map depth (x map ,y map ), when the minimum square error reaches 0, it means that the two values ​​are equal:

[0049] min(||OP||2-depth(x map ,y map )) 2

[0050] =(||(x0,y0,z0)+depth F (e x ,e y ,e z )||2-depth(x map (t),y map (t))) 2

[0051] =(||(x0,y0,z0)+(depth F-1 +t)(e x ,e y ,e z )||2-depth(x map (t),y map (t))) 2

[0052] The constraints are:

[0053]

[0054] In the loss function (x map (t),y map (t)) represents the coordinates of point P in space projected onto the depth map in the direction of OP. According to the parametric equation of point P in space:

[0055]

[0056] And the depth map projected to point P is called map, then the projection point (x map (t),y map(t)) parameter equation is:

[0057]

[0058]

[0059]

[0060] For convenience, (x map (t),y map (t)) The parameter equation can be expressed in a unified form:

[0061]

[0062] Where d = R h / (2tan(π / 3)), represents the focal length, which is the vertical distance from the viewpoint to the depth map. (k x ,b x ,k y ,b y ,k,b) have different values ​​when map takes different depth maps. For example, when map = map2, its value is

[0063] Put (x map (t),y map After the (t)) parameter equation is brought into the loss function, the optimization problem is as follows:

[0064]

[0065] When the minimum value of the objective function is 0, it means that depth(x map ,y map ) is equal to ||OP||2, so take t * for:

[0066] t * =argmin t (||OP||2-depth interp (x map ,y map )) 2

[0067] but That is, the latest depth value is solved. However, due to the error between depth(·,·) and the actual depth, the minimum value of the loss function may not be equal to 0. Therefore, when the value of the loss function is less than a certain threshold ∈, it is considered to reach 0 and t is solvable. In this embodiment, ∈ is 5 cm.

[0068] Step S4, simplifying and optimizing the loss function.

[0069] Among them, the local linear interpolation method is used to interpolate the depth map to simplify the optimization loss function. Specifically, the following steps are included:

[0070] Step S4-1, optimizing the feasible domain of variables.

[0071] In the optimization objective function of step S3-3, depth(x,y) is used to represent the depth value obtained by rendering the depth map at the coordinate (x,y). The depth map rendered by the GPU is at the pixel level. In order to make depth(x,y) continuous and differentiable, this embodiment interpolates the depth. At the same time, in order to make the optimization problem easier to solve, this embodiment uses a linear interpolation method on a two-dimensional plane.

[0072] In order to perform a single interpolation method, it is necessary to further restrict the feasible domain of the variable t. This is because the feasible domain of t is too large, which makes (x map (t),y map (t)) is too long in the depth map, and linear interpolation will have a large error. map (t),y map (t)) must be the same as (x map (0),y map The Chebyshev distance of (0)) must be less than M as a constraint condition, and in this embodiment, M=1.5 cm is generally taken. The formula of the constraint condition is as follows:

[0073]

[0074] Solving the above formula, we can get the range of t:

[0075]

[0076]

[0077] t left =max(min(t x1 ,t x2 ),min(t y1 ,t y2 ))

[0078] t right =min(max(t x1 ,t x2 ),max(t y1 ,t y2 ))

[0079] t∈[t left ,t right ]

[0080] The range t∈[t left ,tright ] also conforms to the fact that the laser depth of two adjacent frames does not change much. Combined with the range of t in step S3-2 t∈[-depth F-1 / N,depth F-1 / N], so the feasible domain of the optimization problem is finally:

[0081]

[0082] For the convenience of representation, the domain of t is uniformly expressed as [t min ,t max ].

[0083] Step S4-2, using a linear interpolation method to simplify the optimization loss function.

[0084] Note (x min ,y min )=(x map (t min ),y map (t min )),(x max ,y max )=(x map (t max ),y map (t max )) and use bilinear interpolation to find the depth of these two points depth(x min ,y min ) and depth(x max ,y max ). Use a linear function to fit the line segment (x min ,y min )-(x max ,y max ), that is, the depth value is along with (x map (t),y map (t)) to (x min ,y min ) changes linearly. map (t),y map (t)) parametric equation, its trajectory is linear, so the interpolation function can be regarded as map (t)-x min or map (t)-y min A linear function, substituted into depth(x min ,y min ) and depth(x max ,y max ) value, the interpolation function is obtained as:

[0085]

[0086] Where Δx=|x max -x min |,Δy=|y max -y min |. The meaning of the above formula is that when Δx is large, use x map (t) is used as the depth interpolation parameter, otherwise y is used map (t) is used as the depth interpolation parameter. The reason for the above approach is to avoid the situation where Δx or Δy is too small, resulting in low interpolation accuracy. It is important that this interpolation method only selects x map (t),y map (t) is used as a variable, the form is simple. map (t) and y map (t) is consistent. For the sake of convenience, depth(·,·) is expressed as Δx>Δy in the following text.

[0087] Figure 4 Schematic diagram of a primary interpolation method in this embodiment.

[0088] The schematic diagram of the quadratic plane linear interpolation method is shown in FIG4 , where the coordinates of the figure represent the pixels of the depth map, and the oblique lines in the figure represent (x map (t),y map (t)) in the depth map, the middle dot represents the point (x map (0),y map (0)), the range represented by the rectangle represents the Chebyshev distance constraint, and the two points on both sides represent (x map (t min ),y map (t min )) and (x map (t max ),y map (t max )), and the line segment in the rectangle indicates that when t∈[t min ,t max ]Time(x map (t),y map (t)), which is exactly the range considered in the optimization problem. It can be seen that due to the distance constraint, the number of pixels in the horizontal and vertical directions of this trajectory is very small and no more than 2M. Therefore, when t∈[t min ,t max ], the use of a single interpolation method will not result in a large error.

[0089] Step S5, finding the optimal solution to the optimization problem.

[0090] Among them, the first-order Taylor expansion is performed on the high-order terms of the optimization loss function to further simplify the loss function, so as to obtain the analytical form of the optimal solution to the optimization problem.

[0091] In this embodiment, Taylor expansion is first performed on the high-order terms.

[0092] The loss function obtained in step S3-3 of the present invention contains a quadratic term ||(x0,y0,z0)+(depth F-1 +t)(e x ,e y ,e z )||2 term. Due to the existence of this term, it is difficult to find the formal solution of the loss function in constant time. Let loss1 = ||(x0,y0,z0)+(depth F-1 +t)(e x ,e y ,e z )||2 and perform Taylor expansion on this term to obtain:

[0093]

[0094] Since the value of t is small, the higher-order o(t 2 ) item, and finally get

[0095]

[0096] Use this approximation The loss function loss1 is used to approximate the solution to the problem. In the unmanned vehicle simulation scenario, the error between the solution to the approximate problem and the solution to the original problem is extremely small. Moreover, in a stationary scenario, the error decreases rapidly and approaches 0 as the number of solutions increases.

[0097] After using Taylor expansion, the approximate form of the problem is:

[0098]

[0099] Secondly, find the analytical form of the optimal solution.

[0100] For the loss function loss similar (t) can be expanded to obtain:

[0101]

[0102] For ease of representation, remember:

[0103]

[0104] Then the optimization problem can be expressed as:

[0105]

[0106] The solution of the above equation must be located at the endpoint t∈{t min ,t max}, or when the partial derivative of loss with respect to t is 0. Next, find loss similar Partial derivative with respect to t:

[0107] For the loss function loss similar (t) can be expanded to obtain:

[0108]

[0109] For ease of representation, remember:

[0110]

[0111] Then the optimization problem can be expressed as:

[0112]

[0113] The solution of the above equation must be located at the endpoint t∈{t min ,t max}, or when the partial derivative of loss with respect to t is 0. Next, find loss similar Partial derivative with respect to t:

[0114]

[0115] Let the above formula be 0, and we can find at most four solutions, denoted as t1, t2, t3, t4:

[0116]

[0117] If Δ1<1 or Δ2<0, there may be less than four solutions, and the set whose derivative is 0 is T. partial , and record the candidate set T candidate ={t min ,t max}∩T partial ,

[0118] Then the minimum loss function can be obtained as:

[0119]

[0120] Step S6, judging whether the optimization loss function reaches the optimization threshold according to the optimal solution.

[0121] Whether the optimization loss function reaches the optimization threshold is detected to determine whether the optimization is successful. If successful, the depth of the latest frame is the sum of the depth of the previous frame and the value of the corresponding optimization variable when the loss function takes the minimum value. Otherwise, the CPU method is used to use the physical engine ray intersection to obtain the depth of the latest frame.

[0122] If loss target Less than the threshold, i.e. loss target ≤∈, then it means that we have found the solution t * , and the solution is:

[0123]

[0124] The depth value of the laser beam in frame F can be obtained

[0125] If loss target >∈, it means that the solution failed due to interpolation error and other reasons, then the CPU method is used to use the physical engine ray intersection to obtain the depth of the latest frame

[0126] Step S7, converting the depth information of the laser radar into a corresponding point cloud.

[0127] Figure 5 It is a point cloud image of a crossroad in an embodiment of the present invention.

[0128] Figure 6 It is a point cloud image of a tunnel scene in an embodiment of the present invention.

[0129] Conversion formula using depth and point coordinates Get in Σ Car The point cloud coordinates in the coordinate system, the point cloud effect is as follows Figure 5 , Figure 6 As shown (plan view of a single laser radar simulation effect).

[0130] This embodiment is based on the characteristics of close distance and small parallax of laser radars in the scenario where a single vehicle is equipped with multiple laser radars, and combines the point cloud information of the previous frame and the depth map information of the latest frame to convert the point cloud of the latest frame into an optimization problem to be solved.

[0131] The optimization loss function is concisely represented using the plane interpolation method, and the Taylor expansion is used to further approximate the optimization problem, expressing the problem as an approximate form of a combination of simple linear terms and fractions about the optimization variables.

[0132] The minimum value of the loss function is calculated using the extreme points and endpoints, and finally the optimization problem formal solution can be found in constant time.

[0133] Functions and Effects of the Embodiments

[0134] According to the GPU-accelerated unmanned driving multi-lidar collaborative simulation method involved in this embodiment, the specific steps are: Step 1, obtain the depth information of the lidar at the initial position of the vehicle, and convert it into a point cloud output of the corresponding lidar, to obtain the depth information and point cloud of the lidar at the initial position; Step 2, based on the latest position of the vehicle, use the GPU to generate a depth map surrounding the vehicle; Step 3, construct an optimization loss function and an optimization problem based on the depth map; Step 4, simplify the optimization loss function; Step 5, find the optimal solution to the optimization problem; Step 6, determine whether the optimization loss function reaches the optimization threshold based on the optimal solution; Step 7, convert the depth information of the lidar into a corresponding point cloud.

[0135] Therefore, the multi-lidar collaborative simulation method based on GPU acceleration proposed in the above embodiment is based on the characteristics of the close distance and small parallax of the laser radars in the scenario where a single vehicle is equipped with multiple laser radars, and fully utilizes the point cloud information of the previous frame of simulation, and adopts GPU and numerical optimization iterative solution method to efficiently simulate multiple laser radars.

[0136] The multi-lidar collaborative simulation method based on GPU acceleration proposed in the above embodiment efficiently utilizes GPU and CPU computing resources, especially for the simulation tasks of a single vehicle equipped with multiple lidar scenarios. It can perform efficient collaborative simulation of multiple lidars, and while ensuring high-efficiency simulation, it can generate high-quality point clouds to ensure that the point clouds are not distorted.

[0137] The above-mentioned embodiments are preferred examples of the present invention and are not intended to limit the protection scope of the present invention.

Claims

1. A GPU-accelerated unmanned driving multi-lidar collaborative simulation method, characterized in that: The following steps are involved: Step 1, obtain the depth information of the laser radar at the initial position of the vehicle, and convert it into the point cloud output of the corresponding laser radar to obtain the depth information and point cloud of the laser radar at the initial position; Step 2: Based on the latest position of the vehicle, in the latest frame of the simulation process, according to the latest position of the vehicle, a set of high-resolution depth maps around the vehicle are generated using the GPU with the center of the vehicle as the viewpoint. The depth maps are three, with ΣCar as the reference system and the vehicle as the viewpoint, and are rotated around the X-axis at angles of 0, 2π / 3, and 4π / 3 respectively to generate depth maps with a horizontal viewing angle FOV of 2π / 3; Step 3, constructing an optimization loss function and an optimization problem based on the depth map, based on the depth corresponding to the previous frame of the laser beam, taking the depth difference between the two frames of laser radar as the optimization variable, and combining the depth information of the depth map of the latest frame to construct an optimization loss function, and constructing it into a constrained optimization problem; Step 4, simplifying the optimization loss function; Step 5, finding the optimal solution to the optimization problem; Step 6, judging whether the optimization loss function reaches the optimization threshold according to the optimal solution; Step 7: Convert the depth information of the laser radar into a corresponding point cloud.

2. The GPU-accelerated unmanned driving multi-lidar collaborative simulation method according to claim 1, characterized in that: in, In step 1, in the first frame of the simulation process, the CPU method is used to simulate each laser beam of each lidar through the physical engine ray intersection, and the depth information relative to the lidar sensed by the laser beam at the initial position of the vehicle is obtained, and converted into the point cloud output of the corresponding lidar, and the azimuth and angle of each laser beam are recorded.

3. According to the GPU-accelerated unmanned driving multi-lidar collaborative simulation method of claim 1, Features: Among them, in step 3, based on the depth corresponding to the previous frame of the laser beam, the depth difference between the two frames of laser radar is used as the optimization variable, and the depth information of the depth map of the latest frame is combined to construct the optimization loss function, which is constructed into a constrained optimization problem, specifically including the following steps: Step 3-1, analyzing the assumptions and rationality of the optimization problem; Step 3-2, setting the optimization variables of the optimization problem; Step 3-3, optimizing the optimization loss function.

4. The GPU-accelerated unmanned driving multi-lidar collaborative simulation method according to claim 1, characterized in that: in, In step 4, the depth map is interpolated using a local linear interpolation method to simplify the optimization loss function, which specifically includes the following steps: Step 4-1, optimize the feasible domain of variables; Step 4-2, using a linear interpolation method to simplify the optimization loss function.

5. The GPU-accelerated unmanned driving multi-lidar collaborative simulation method according to claim 1, characterized in that: in, In step 5, a first-order Taylor expansion is performed on the high-order terms of the optimization loss function to further simplify the loss function, thereby obtaining an analytical form of the optimal solution to the optimization problem.

6. The GPU-accelerated unmanned driving multi-lidar collaborative simulation method according to claim 1, characterized in that: in, In step 6, it is detected whether the optimization loss function reaches the optimization threshold to determine whether the optimization is successful. If successful, the depth of the latest frame is the sum of the depth of the previous frame and the value of the corresponding optimization variable when the loss function takes the minimum value. Otherwise, the CPU method is used to use the physical engine ray intersection to obtain the depth of the latest frame.

Images