LiDAR Point Cloud Simulation Method, Device, and Computer Equipment

Abstract

The present application relates to a method, apparatus and computer device for lidar point cloud simulation. The method includes: a first processor obtains the position and pose of a target object; a second processor determines first mesh data of the target object based on the position and pose; the second processor determines second mesh data from the first mesh data based on a target scanning range of the lidar; the second processor determines target polygons intersecting with laser beams in the laser beam data within the target scanning range and intersection point coordinates intersecting with the laser beams in the second mesh data; the second processor calculates the reflected point cloud intensity based on the direction vector information of the laser beams, the normal vector information of the target polygons and the intersection point coordinates, and the second processor determines the point cloud data corresponding to the target object within the target scanning range based on the point cloud intensity and the intersection point coordinates. Using this method can improve the simulation efficiency of lidar point cloud.

Description

Technical Field

[0001] This application relates to the technical field of sensor simulation, and particularly to a method, device, and computer equipment for lidar point cloud simulation. Background Art

[0002] Lidar can provide accurate environmental depth measurement, so it is widely used in environmental perception and positioning tasks. Lidar is an important part of the autonomous driving sensor system, and the simulation of point cloud data is an indispensable part of the data closed-loop of the lidar perception algorithm.

[0003] The existing lidar point cloud simulation technologies are mainly divided into two categories. One is the GPU rendering method based on game engines, and the other is the CPU-based simulation method. The first method usually uses a mature large game engine to render the point cloud, and the second method uses the CPU to simulate the point cloud. Since neither of them considers the characteristic that lidar, especially the most widely used mechanical lidar at present, emits multiple beams (up to 64 or even 128 beams) simultaneously, there are problems of low simulation efficiency and high cost for lidar point cloud. Summary of the Invention

[0004] Based on this, it is necessary to provide a lidar point cloud simulation method, device, and computer equipment that can improve the simulation efficiency of lidar point cloud and reduce costs for the above technical problems.

[0005] In a first aspect, this application provides a lidar point cloud simulation method. The method includes:

[0006] A first processor obtains the position and attitude of a target object;

[0007] A second processor determines first mesh data of the target object based on the position and the attitude;

[0008] The second processor determines second mesh data from the first mesh data based on a target scanning range of the lidar;

[0009] The second processor determines target polygons that intersect with laser beams in the laser beam data within the target scanning range in the second mesh data, and intersection point coordinates that intersect with the laser beams;

[0010] The second processor calculates the reflected point cloud intensity based on the direction vector information of the laser beam, the normal vector information of the target polygon, and the intersection point coordinates, and

[0011] The second processor determines point cloud data corresponding to the target object within the target scanning range based on the point cloud intensity and the intersection point coordinates.

[0012] In one embodiment, before determining the second grid data from the first grid data for the target scanning range based on lidar, the method further includes:

[0013] Dividing the preset frame time according to a preset number of divisions to obtain a target scanning time; the preset frame time is the time required for the lidar to scan one week;

[0014] Determining a scanning range sequence based on the preset number of divisions and the target scanning time;

[0015] Selecting a scanning range from the scanning range sequence in sequence order as the target scanning range.

[0016] In one embodiment, determining the point cloud data corresponding to the target object within the target scanning range based on the point cloud intensity and the intersection point coordinates includes:

[0017] Whenever a target scanning range is selected from the scanning range sequence, determining the point cloud data corresponding to the target object within the selected target scanning range based on the point cloud intensity and the intersection point coordinates, until the point cloud data corresponding to the target objects within each scanning range in the scanning range sequence is obtained;

[0018] The method further includes: combining the point cloud data corresponding to the target objects within each scanning range in the scanning range sequence into the point cloud simulation data of the target object.

[0019] In one embodiment, before determining the target polygon that intersects the laser beam data within the target scanning range and the intersection point coordinates of the intersection with the laser beam in the second grid data, the method further includes:

[0020] Obtaining the starting point and direction vector information of the laser beam within the target scanning range, and the normal vector information of the target polygon;

[0021] Calculating the distance from the starting point to the intersection point based on the starting point, the direction vector information, and the normal vector information of the target polygon; the intersection point is the point where the target polygon intersects the laser beam;

[0022] Determining the target polygon that intersects the laser beam in the laser beam data and the intersection point coordinates of the intersection with the laser beam in the second grid data includes:

[0023] When the distance is less than the effective detection distance of the lidar, it is determined that the laser beam in the laser beam data intersects the target polygon;

[0024] Calculate the intersection coordinates of the intersection point based on the starting coordinates, the distance, and the direction vector information.

[0025] In one embodiment, calculating the reflected point cloud intensity based on the direction vector information of the laser beam, the normal vector information of the target polygon, and the intersection coordinates includes:

[0026] Determine the included angle based on the direction vector information of the laser beam and the normal vector information of the target polygon;

[0027] Calculate the reflected point cloud intensity based on the included angle, the intersection coordinates, and the starting point.

[0028] In one embodiment, the method further includes:

[0029] Determine the calculation block capacity according to the preset number of beam lines and the preset number of target polygons;

[0030] Determine the number of calculation blocks according to the number of laser beam lines, the number of target polygons, and the calculation block capacity;

[0031] Determine the number of threads in each calculation block according to the calculation block capacity;

[0032] Calculating the reflected point cloud intensity based on the direction vector information of the laser beam, the normal vector information of the target polygon, and the intersection coordinates includes:

[0033] Parallelly calculate the direction vector information of the laser beam, the normal vector information of the target polygon, and the intersection coordinates according to the number of calculation blocks and the number of threads to obtain the reflected point cloud intensity.

[0034] In a second aspect, the present application further provides a lidar point cloud simulation device. The device includes a memory and a processor. The memory stores a computer program. The processor includes a first processor and a second processor. The feature is that when the processor executes the computer program, it implements the lidar point cloud simulation method, and the method includes:

[0035] The first processor obtains the position and attitude of the target object;

[0036] The second processor determines the first mesh data of the target object based on the position and the attitude;

[0037] The second processor determines the second mesh data from the first mesh data based on the target scanning range of the lidar;

[0038] The second processor determines a target polygon intersecting with the laser beam lines in the laser beam line data within the target scanning range and the intersection coordinates of the intersection with the laser beam lines in the second grid data;

[0039] The second processor calculates the reflected point cloud intensity based on the direction vector information of the laser beam line, the normal vector information of the target polygon, and the intersection coordinates, and

[0040] The second processor determines the point cloud data corresponding to the target object within the target scanning range based on the point cloud intensity and the intersection coordinates.

[0041] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0042] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0043] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0044] The above lidar point cloud simulation method, device, and computer device determine the corresponding second grid data and laser beam line data through the divided target scanning range, determine the triangles intersecting with the laser beam lines in the second grid data and the intersection coordinates of the intersection with the laser beam lines, calculate the reflected point cloud intensity based on the direction vector information of the laser beam line, the normal vector information of the target polygon, and the intersection coordinates, so as to obtain the point cloud data of the target object; realizing the screening of the grid data and laser beam lines according to the characteristics of the mechanical lidar beam, and combining the parallel computing architecture of the second processor to determine the calculation scheme of the reflected point cloud intensity, effectively improving the simulation efficiency of the lidar point cloud and reducing the simulation cost of the lidar point cloud. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is an application environment diagram of the lidar point cloud simulation method in an embodiment;

[0046] Figure 2 It is a flow schematic diagram of the lidar point cloud simulation method in an embodiment;

[0047] Figure 3 It is a schematic diagram of a calculation block in an embodiment;

[0048] Figure 4 Schematic diagram of a thread in one embodiment;

[0049] Figure 5 Flow schematic diagram of a lidar point cloud simulation method in another embodiment;

[0050] Figure 6 Flow schematic diagram of the steps for calculating the intersection point coordinates in one embodiment;

[0051] Figure 7 Structural block diagram of a lidar point cloud simulation device in one embodiment;

[0052] Figure 8 Structural block diagram of a lidar point cloud simulation device in another embodiment;

[0053] Figure 9 Internal structure diagram of a computer device in one embodiment. Detailed implementation manners

[0054] In order to make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] The lidar point cloud simulation method provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 . Among them, the terminal 102 communicates with the server 104 through a network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed in the cloud or other network servers.

[0056] The present application can be applied to the terminal 102 or the server 104. Taking the terminal 102 as an example, the terminal 102 includes a first processor and a second processor. The first processor obtains the position and attitude of the target object; the second processor determines the first mesh data of the target object based on the position and attitude; the second processor determines the second mesh data from the first mesh data based on the target scanning range of the lidar; the second processor determines the target polygon that intersects with the laser beam in the laser beam data within the target scanning range, and the intersection point coordinates that intersect with the laser beam; the second processor calculates the reflected point cloud intensity based on the direction vector information of the laser beam, the normal vector information of the target polygon, and the intersection point coordinates; and the second processor determines the point cloud data corresponding to the target object within the target scanning range based on the point cloud intensity and the intersection point coordinates.

[0057] CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model. By leveraging the processing power of the graphics processing unit (GPU), it can significantly improve computing performance.

[0058] Among them, the terminal 102 can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The terminal 102 is equipped with a mechanical lidar sensor system. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.

[0059] In one embodiment, as Figure 2 shown, a lidar point cloud simulation method is provided. This method can be used in the terminal 102 or the server 104. Taking the application of this method to the Figure 1 terminal 102 as an example for illustration, it includes the following steps:

[0060] S202, the first processor obtains the position and pose of the target object.

[0061] Among them, the first processor can be a CPU (central processing unit). The target object can refer to the object scanned by the mechanical lidar. The target object can be a moving or stationary object. For example, the target object can be a pedestrian, a vehicle, a building, etc. The position can refer to the coordinates of the target object. For example, the position of the target object A can be (10.0, 20.0, 0.3). The pose can refer to the direction in which the target object is located. The pose can be represented in the form of (roll angle, pitch angle, yaw angle) and the angle range is from 0 to 2Π. For example, the pose of the target object A is (0.01, 0.04, 3Π / 2). It should be noted that the mechanical lidar can be installed on the top of the terminal. For example, it can be installed on the roof of an autonomous vehicle and is part of the autonomous driving sensor.

[0062] In one embodiment, before S202, the first processor can obtain the historical grid data of the target object in the environmental map, as well as the motion trajectories of the terminal and the target object.

[0063] Among them, the historical grid data can refer to the grid data before a preset moment. The historical grid data can be composed of polygons, and each vertex in the polygon can be represented by coordinates. A target object can be composed of one or more polygons. The preset moment can refer to the moment when the lidar point cloud simulation starts pre-set by the user. For example, in the grid corresponding to target object A, the target object can be composed of 300 triangles. The running trajectory can refer to the identification, position, and attitude of the terminal or target object at different moments. For example, at the moment of 14:51, the identification of target object B is 001, the position is (20.1, 31.3, 0.5), and the attitude is (0.03, 0.0, 3Π / 2).

[0064] Specifically, the first processor can obtain the position and attitude of the target object at the preset moment. The first processor can also receive a lidar point cloud simulation instruction and use the position and attitude stored in the terminal as the position and attitude of the target object. For example, the user can generate a lidar point cloud simulation instruction through a trigger operation. When the first processor receives this lidar point cloud simulation instruction, the first processor can use the position and attitude corresponding to the user identification stored in the terminal as the position and attitude of the target object.

[0065] In one embodiment, S202 includes determining a first position and a first attitude corresponding to a first moment, and a second position and a second attitude corresponding to a second moment in the motion trajectory; the first moment and the second moment are moments adjacent to the preset moment during the motion of the target object; determining a moment parameter based on the preset moment, the first moment, and the second moment; determining the position of the target object based on the moment parameter, the first position, and the second position; and determining the attitude of the target object based on the moment parameter, the first attitude, and the second attitude.[[ID=~7]]

[0066] Among them, the first moment can be the moment before the preset moment and is included in the motion trajectory of the target object. The first position can refer to the position of the target object corresponding to the first moment. The first attitude can refer to the attitude of the target object corresponding to the first moment. The second moment can be the moment after the preset moment and is included in the motion trajectory of the target object. The second position can refer to the position of the target object corresponding to the second moment. The second attitude can refer to the attitude of the target object corresponding to the second moment. The moment parameter can refer to the parameter determined based on the preset moment, the first moment, and the second moment, and this moment parameter can be used to determine the position and attitude of the target object.

[0067] For example, the preset moment is t. In the motion trajectory of the target object, the first moment adjacent to the preset moment is s1.timestamp, and the second moment is s2.timestamp. Then the calculation formula for the moment parameter ratio can be:

[0068]

[0069] The first position corresponding to the first moment s1.timestamp is (s1.x, s1.y, s1.z) and the first attitude is (s1.roll, s1.pitch, s1.yaw). The second position corresponding to the second moment s2.timestamp is (s2.x, s2.y, s2.z) and the second attitude is (s2.roll, s2.pitch, s2.yaw).

[0070] The calculation formula for the position (x, y, z) of the target object can be:

[0071] x = (1 - ratio) * s1.x + ratio * s2.x

[0072] y = (1 - ratio) * s1.y + ratio * s2.y

[0073] z = (1 - ratio) * s1.z + ratio * s2.z

[0074] The calculation formula for the attitude (roll, pitch, yaw) of the target object can be:

[0075] roll = (1 - ratio) * s1.roll + ratio * s2.roll

[0076] pitch = (1 - ratio) * s1.pitch + ratio * s2.pitch

[0077] yaw = (1 - ratio) * s1.yaw + ratio * s2.yaw

[0078] For example, user A logs in to the autonomous driving sensor system through the page. The system can obtain the positions and attitudes of all target objects uploaded by this user A according to the user identification, that is, user A. At the same time, user A can also select the positions and attitudes that meet their own needs among multiple preset moments displayed on the page. Then, the first processor can filter out the positions and attitudes of the target objects corresponding to the preset moments in response to the preset moment selection operation of user A.

[0079] S204. The second processor determines the first mesh data of the target object based on the position and attitude.

[0080] Among them, the second processor may be a CUDA-based GPU. The first mesh data may refer to one or more polygon data corresponding to the target object. For example, the first mesh data corresponding to target object A and target object B are respectively {(10.2, 20.1, 0.3), (11.1, 19.0, 0.4), (12.0, 19.5, 0.2);... (30.0, 50.0, 0.60), (31.0, 51.0, 0.9), (28.0, 45.0, 1.0)} and {(4.0, 6.0, 0.3), (6.0, 3.0, 0.5), (5.0, 4.0, 0.3);... (6.0, 4.0, 0.7), (8.0, 5.7, 0.8), (9.3, 6.0, 1.8)}.

[0081] Specifically, the second processor may obtain historical mesh data from the first processor; and determine the first mesh data of the target object based on the position, pose, and historical mesh data.

[0082] For example, the historical mesh data is mesh_h, the position of the target object is (x, y, z), and the pose is (α, β, γ). The calculation formula for the first mesh data new1_mesh may be:

[0083] new1_mesh = R(α, β, γ) * mesh_h + (x, y, z)

[0084] Among them, the pose parameter R(α, β, γ) may be

[0085]

[0086] S206. The second processor determines the second mesh data from the first mesh data based on the target scanning range of the lidar.

[0087] Among them, the lidar may refer to a mechanical lidar. The target scanning range may refer to the range in which the target to obtain point cloud data is selected within the range formed by one full scan of the lidar. For example, the target scanning range may be the scanning range formed when the lidar scanning angle is 2 degrees. The second mesh data may refer to the data in the first mesh data that belongs to the target scanning range.

[0088] In one embodiment, before S206, the second processor divides the preset frame time according to the preset division quantity to obtain the target scanning time; determines the scanning range sequence based on the preset division quantity and the target scanning time; and selects the scanning range as the target scanning range in sequence order from the scanning range sequence.

[0089] Among them, the preset division quantity can refer to the quantity of divisions of the preset frame time set in advance. For example, the preset division quantity can be 300. The preset frame time is the time required for the lidar to scan one week. For example, the preset frame time can be 100 milliseconds. The target scan time can refer to the time of preset frame time / preset division quantity after dividing the preset frame time into the preset division quantity. For example, if the preset division quantity is 300 and the preset frame time is 100 milliseconds, then the target scan time is 100 milliseconds / 300, that is, 0.333 milliseconds. The scan range sequence can refer to the sequence number of the scan range sequence determined by the preset division quantity. The target scan time determines the scan time of the lidar corresponding to the scan range where the point cloud data to be determined in the scan range sequence is located. For example, if the preset division quantity is 300 and the target scan time is 0.333 milliseconds, then the corresponding scan range sequence can be {scan range 1, scan range 2,... scan range 299, scan range 300}, and the scan time corresponding to each scan range is 0.333 milliseconds.

[0090] Specifically, the second processor can filter the first grid data according to the target scan range of the lidar to obtain the second grid data belonging to the target scan range in the first grid data.

[0091] In one embodiment, the second processor takes the lidar center as the origin, establishes a spherical coordinate system, and filters all triangles of all the first grid data in the entire environment according to the horizontal angle of the beam (target scan range). Triangles exceeding the horizontal angle range of the beam can be ignored, and triangles within the horizontal angle range are saved to obtain the second grid data.

[0092] S208. The second processor determines the target polygons intersecting with the laser beams in the laser beam data within the target scan range in the second grid data, and the intersection coordinates of the intersections with the laser beams.

[0093] Among them, the laser beam data can refer to the data related to the laser beam. The data related to the laser beam can be the starting point of the laser beam, direction vector information, etc.

[0094] Specifically, the second processor can obtain the laser beam data within the target scan range. The second processor can also receive the lidar point cloud simulation instruction and use the laser beam data stored in the terminal as the laser beam data within the target scan range.

[0095] Among them, the laser beam bundle can refer to the laser beam bundle emitted by the lidar. When rotating, a mechanical lidar can emit 64 laser beam bundles each time and continuously emit 6 times. The target polygon can refer to the polygon in the second grid data that intersects with the laser beam bundle. For example, the target polygon can be a triangle, a rectangle, etc. The intersection point coordinates can refer to the coordinates corresponding to the intersection points between the laser beam bundle and the target polygon. For example, the intersection point coordinates can be (10.0, 20.0, 0.3).

[0096] In one embodiment, before S208, the second processor acquires the starting point and direction vector information of the laser beam bundle, as well as the normal vector information of the target polygon; calculates the distance from the starting point to the intersection point based on the starting point, direction vector information, and normal vector information of the target polygon.

[0097] In one embodiment, S208 includes that when the distance is less than the effective detection distance of the lidar, the second processor determines that the laser beam bundle in the laser beam bundle data intersects with the target polygon; calculates the intersection point coordinates of the intersection point based on the starting coordinates, distance, and direction vector information.

[0098] S210, the second processor calculates the reflected point cloud intensity based on the direction vector information of the laser beam bundle, the normal vector information of the target polygon, and the intersection point coordinates, and the second processor determines the point cloud data corresponding to the target object within the target scanning range based on the point cloud intensity and the intersection point coordinates.

[0099] Among them, the reflected point cloud intensity can refer to the reflected point cloud intensity corresponding to the intersection point coordinates. The point cloud data can refer to the data in which the intersection point coordinates and the point cloud intensity correspond to each other.

[0100] In one embodiment, S210 includes that the second processor determines the included angle based on the direction vector information of the laser beam bundle and the normal vector information of the target polygon; calculates the reflected point cloud intensity based on the included angle, intersection point coordinates, and starting point.

[0101] Among them, the included angle can be the included angle between the direction vector of the laser beam bundle and the normal vector of the target polygon.

[0102] For example, the intersection point coordinates are P = O + t * D, where O is the starting coordinate of the laser beam bundle, t is the distance from the starting point of the laser beam bundle to the intersection point (a fixed value), and D is the direction vector information of the corresponding laser beam bundle. The normal vector information of the target polygon is N(A, B, C). Then the included angle is θ, and the cosine of the included angle is cosθ = D * N(A, B, C). The point cloud intensity = reflectivity * (1 - cosθ) 0.5 * exp(-1.0 * t), and the reflectivity can be 1.

[0103] In one embodiment, before S210, the second processor determines the calculation block capacity according to the preset number of laser beams and the preset number of target polygons; determines the number of calculation blocks according to the number of laser beams, the number of target polygons, and the calculation block capacity; and determines the number of threads in each calculation block according to the calculation block capacity.

[0104] Among them, the preset number of laser beams can refer to the preset number of laser beams, which is used to represent the calculation capacity of the calculation block for laser beams. For example, the preset number of laser beams can be 16. The preset number of target polygons can refer to the preset number of target polygons, which is used to represent the calculation capacity of the calculation block for target polygons. For example, the preset number of polygons can be 256. The calculation block capacity is used to represent the calculation capacity of the calculation block for laser beams and target polygons. For example, the calculation block capacity can be 16 laser beams and 256 target polygons. The number of laser beams can refer to the number of laser beams in the target scanning range. The number of target polygons can refer to the number of target polygons in the target scanning range. Figure 3 FIG. is a schematic diagram of a calculation block in one embodiment; the number of calculation blocks can refer to the number of calculation blocks. The calculation block can be used to judge the target polygons intersecting with the laser beams and calculate the reflected point cloud intensity. Figure 4 FIG. is a schematic diagram of a thread in one embodiment; the number of threads can refer to the number of threads included in the calculation block. For example, 1 calculation block includes 256 threads, and each thread can be used to judge whether 1 triangle intersects with 16 laser beams respectively.

[0105] In one embodiment, S210 includes the second processor parallelly calculating the direction vector information of the laser beams, the normal vector information of the target polygons, and the intersection coordinates according to the number of calculation blocks and the number of threads to obtain the reflected point cloud intensity.

[0106] In one embodiment, S210 includes that whenever the second processor selects a target scanning range from the scanning range sequence, it determines the point cloud data corresponding to the target object in the selected target scanning range based on the point cloud intensity and the intersection coordinates until the point cloud data corresponding to the target objects in each scanning range in the scanning range sequence is obtained.

[0107] In one embodiment, after S210, the second processor combines the point cloud data corresponding to the target objects in each scanning range in the scanning range sequence into the point cloud simulation data of the target object.

[0108] For example, Figure 5Schematic flow chart of the lidar point cloud simulation method in another embodiment; as shown in the figure, first, it is judged whether a frame is finished, that is, it is judged whether the lidar has scanned a full circle. If not, the positions and postures of the host vehicle and all objects are updated. After the mesh data of the objects is rotated and translated, the lidar emits 64×6 lidar beams. The triangles in the mesh data are filtered according to the average horizontal angle of the beams. The intersection calculation is performed between the lidar and the filtered triangles, and the point cloud coordinates (intersection coordinates) and point cloud intensity are calculated. After saving the data, the time is increased by 1 / 300 frame. When a frame is finished, the frame number is automatically incremented by 1. When the frame number is greater than the preset frame number m, the process ends.

[0109] In the above lidar point cloud simulation method, the corresponding second mesh data and lidar beam data are determined through the divided target scanning range, and the triangles intersecting with the lidar beams in the second mesh data and the intersection coordinates of the intersections with the lidar beams are determined. Based on the direction vector information of the lidar beams, the normal vector information of the target polygon, and the intersection coordinates, the reflected point cloud intensity is calculated, so as to obtain the point cloud data of the target object; it realizes the screening of the mesh data and lidar beams according to the characteristics of the mechanical lidar beams, and combines the parallel computing architecture of the GPU to determine the scheme for calculating the reflected point cloud intensity, effectively improving the simulation efficiency of the lidar point cloud and reducing the cost of lidar point cloud simulation.

[0110] In one embodiment, as Figure 6 shown, the steps of calculating the intersection coordinates include:

[0111] S602, obtain the starting point and direction vector information of the lidar beam in the target scanning range, and the normal vector information of the target polygon.

[0112] Specifically, the second processor can obtain the starting point and direction vector information of the lidar beam in the target scanning range, and the normal vector information of the target polygon. The second processor can also receive the lidar point cloud simulation instruction, and use the starting point, direction vector information, and normal vector information stored in the terminal as the starting point and direction vector information of the lidar beam, and the normal vector information of the target polygon.

[0113] S604, calculate the distance from the starting point to the intersection based on the starting point and direction vector information, and the normal vector information of the target polygon; the intersection is the point where the target polygon intersects with the lidar beam.

[0114] Among them, the starting point of the laser beam can refer to the starting point of the emission of the laser beam. For example, the starting point can be (4.0, 5.0, 2.0). The direction vector information of the laser beam can be used to represent the direction of the laser beam. For example, the direction vector information of the laser beam can be (0.612, 0.612, 0.56). The normal vector information of the target polygon can refer to the vector information perpendicular to the plane where the target polygon is located. For example, the normal vector information of the target polygon can be (0.745, 0.579, 0.331). The distance can refer to the distance from the starting point of the laser beam to the intersection point. The intersection point is the point where the target polygon intersects with the laser beam.

[0115] For example, the laser beam R1 can be expressed as R1 = O + t * D; where O is the starting coordinate of the laser beam, t is the distance from the starting point of the laser beam to the intersection point, and D is the direction vector information of the corresponding laser beam. The vertex coordinates of the polygon are A(A1, A2, A3), B(B1, B2, B3), and C(C1, C2, C3) respectively.

[0116] The calculation formula for the normal vector information of the polygon can be:

[0117] N(A, B, C) = (B - A) × (C - A) / (|B - A| * |C - A|)

[0118] Then the calculation formula for the distance t is

[0119] S606. When the distance is less than the effective detection distance of the lidar, it is determined that the laser beam in the laser beam data intersects with the target polygon.

[0120] Among them, the effective detection distance of the lidar can refer to the distance at which the lidar can effectively detect the environment. For example, the effective detection distance can be 200 meters. The intersection point coordinates can refer to the intersection point coordinates of the laser beam and the target polygon.

[0121] In one embodiment, if the calculated distance is less than zero, the laser beam does not intersect with the polygon.

[0122] For example, if the second processor determines that the distance is less than 200 meters, it is determined that the laser beam in the laser beam data intersects with the target polygon.

[0123] S608. Calculate the intersection point coordinates of the intersection point based on the starting coordinates, distance, and direction vector information.

[0124] In this embodiment, by calculating the distance from the starting point to the intersection point based on the starting point, direction vector information, and normal vector information of the target polygon, when the distance is less than the effective detection distance of the lidar, it is determined that the laser beam in the laser beam data intersects the target polygon, and the intersection coordinates of the intersection point are calculated based on the starting coordinates, distance, and direction vector information. This can improve the simulation efficiency of lidar point clouds and reduce the simulation cost of lidar point clouds.

[0125] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0126] Based on the same inventive concept, an embodiment of the present application further provides a lidar point cloud simulation device for implementing the lidar point cloud simulation method involved above. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the lidar point cloud simulation device provided below can refer to the limitations on the lidar point cloud simulation method in the above text, and will not be repeated here.

[0127] In one embodiment, as Figure 7 shown, a lidar point cloud simulation device is provided, including: a first acquisition module 702, a first determination module 704, a third determination module 706, a calculation and determination module 708, and a calculation and determination module 710, where:

[0128] The first acquisition module 702 is used for the first processor to acquire the position and attitude of the target object;

[0129] The first determination module 704 is used for the second processor to determine the first grid data of the target object based on the position and attitude;

[0130] The second determination module 706 is used for the second processor to determine the second grid data from the first grid data based on the target scanning range of the lidar;

[0131] A third determination module 708, configured to determine, by a second processor, a target polygon intersecting a laser beam in the laser beam data within a target scanning range and intersection coordinates of the intersection with the laser beam in the second grid data;

[0132] A calculation and determination module 710, configured to calculate, by the second processor, the reflected point cloud intensity based on the direction vector information of the laser beam, the normal vector information of the target polygon, and the intersection coordinates, and determine, by the second processor, the point cloud data corresponding to the target object within the target scanning range based on the point cloud intensity and the intersection coordinates.

[0133] In one embodiment, the first acquisition module 702 is further configured to acquire the motion trajectory of the target object; determine a first position and a first pose corresponding to a first moment, and a second position and a second pose corresponding to a second moment in the motion trajectory; the first moment and the second moment are moments adjacent to a preset moment during the motion of the target object; determine a moment parameter based on the preset moment, the first moment, and the second moment; determine the position of the target object based on the moment parameter, the first position, and the second position; and determine the pose of the target object based on the moment parameter, the first pose, and the second pose.

[0134] In one embodiment, the second determination module 706 is configured to acquire historical grid data; and determine the first grid data of the target object based on the position, the pose, and the historical grid data.

[0135] In one embodiment, the calculation and determination module 710 is further configured to, whenever a target scanning range is selected from the scanning range sequence, determine the point cloud data corresponding to the target object within the selected target scanning range based on the point cloud intensity and the intersection coordinates until the point cloud data corresponding to the target object within each scanning range in the scanning range sequence is obtained; and combine the point cloud data corresponding to the target object within each scanning range in the scanning range sequence into the point cloud simulation data of the target object.

[0136] In one embodiment, the third determination module 708 is further configured to acquire the starting point and direction vector information of the laser beam, and the normal vector information of the target polygon; calculate the distance from the starting point to the intersection point based on the starting point, the direction vector information, and the normal vector information of the target polygon; the intersection point is the point where the target polygon intersects the laser beam; when the distance is less than the effective detection distance of the lidar, it is determined that the laser beam in the laser beam data intersects the target polygon; and calculate the intersection coordinates of the intersection point based on the starting coordinates, the distance, and the direction vector information.

[0137] In one embodiment, the calculation and determination module 710 is further configured to determine an included angle based on the direction vector information of the laser beam and the normal vector information of the target polygon; and calculate the reflected point cloud intensity based on the included angle, the intersection coordinates, and the starting point.

[0138] In one embodiment, the calculation and determination module 710 is further configured to determine the calculation block capacity according to the preset number of wire harnesses and the preset number of target polygons; determine the number of calculation blocks according to the number of laser wire harnesses, the number of target polygons, and the calculation block capacity; determine the number of threads in each calculation block according to the calculation block capacity; and perform parallel calculation on the direction vector information of the laser wire harness, the normal vector information of the target polygon, and the intersection coordinates according to the number of calculation blocks and the number of threads to obtain the reflected point cloud intensity.

[0139] In one embodiment, as Figure 8 shown, the apparatus further includes: a selection module 712, where

[0140] The selection module 712 is configured to divide the preset frame time according to the preset division number to obtain the target scan time; the preset frame time is the time required for the lidar to scan one week; determine the scan range sequence according to the preset division number and the target scan time; and select the scan range as the target scan range from the scan range sequence in the sequence order.

[0141] In the above embodiment, the corresponding second grid data and laser wire harness data are determined through the divided target scan range, and the triangles intersecting the laser wire harness in the second grid data and the intersection coordinates intersecting the laser wire harness are determined. Based on the direction vector information of the laser wire harness, the normal vector information of the target polygon, and the intersection coordinates, the reflected point cloud intensity is calculated, so as to obtain the point cloud data of the target object; it realizes the screening of the grid data and the laser wire harness according to the characteristics of the mechanical lidar wire harness, and combines the parallel computing architecture of the GPU to determine the scheme for calculating the reflected point cloud intensity, effectively improving the simulation efficiency of the lidar point cloud and reducing the simulation cost of the lidar point cloud.

[0142] Each module in the above lidar point cloud simulation apparatus can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in the form of hardware or independent of the processor, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0143] In one embodiment, another lidar point cloud simulation apparatus is further provided. The apparatus includes a memory and a processor. The memory stores a computer program. The processor includes a first processor and a second processor. The feature is that when the processor executes the computer program, it realizes the lidar point cloud simulation method. The method includes:

[0144] The first processor acquires the position and attitude of the target object;

[0145] The second processor determines the first grid data of the target object based on the position and attitude;

[0146] The second processor determines second grid data from the first grid data based on the target scanning range of the lidar;

[0147] The second processor determines target polygons that intersect the laser beams in the laser beam data within the target scanning range in the second grid data, and the intersection coordinates of the intersections with the laser beams;

[0148] The second processor calculates the reflected point cloud intensity based on the direction vector information of the laser beam, the normal vector information of the target polygon, and the intersection coordinates, and the second processor determines the point cloud data corresponding to the target object within the target scanning range based on the point cloud intensity and the intersection coordinates.

[0149] In one embodiment, a computer device is provided. The computer device can be a terminal or a server. In this embodiment, the computer device is taken as an example of a terminal for illustration, and its internal structure diagram can be as Figure 9 shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (Near Field Communication), or other technologies. The computer program, when executed by the processor, implements a lidar point cloud simulation method. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0150] Those skilled in the art can understand that Figure 9 the structure shown in

[0151] In one embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the above-mentioned embodiments are implemented.

[0152] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned embodiments are implemented.

[0153] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the above-mentioned embodiments are implemented.

[0154] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards of relevant countries and regions.

[0155] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memories. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, data processing logics based on quantum computing, etc., without limitation.

[0156] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope described in this specification.

[0157] The above embodiments only represent several implementation manners of the present application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method for simulating lidar point cloud, characterized in that, The method includes: The first processor obtains the position and pose of the target object; The second processor determines the first mesh data of the target object based on the position and the pose; The second processor determines the second mesh data from the first mesh data based on the target scanning range of the lidar; The second processor obtains the starting point and direction vector information of the laser beam line within the target scanning range, and the normal vector information of the target polygon; calculates the distance from the starting point to the intersection point based on the starting point, the direction vector information, and the normal vector information of the target polygon; the intersection point is the point where the target polygon intersects with the laser beam line; when the distance is less than the effective detection distance of the lidar, it is determined that the laser beam line in the laser beam line data intersects with the target polygon; calculates the intersection point coordinates of the intersection point based on the starting coordinates, the distance, and the direction vector information; The second processor calculates the reflected point cloud intensity based on the direction vector information of the laser beam line, the normal vector information of the target polygon, and the intersection point coordinates, and The second processor determines the point cloud data corresponding to the target object within the target scanning range based on the point cloud intensity and the intersection point coordinates.

2. The method according to claim 1, wherein Before obtaining the position and pose of the target object, the method further includes: Obtaining the motion trajectory of the target object; The obtaining the position and pose of the target object includes: Determining a first position and a first pose corresponding to a first moment, and a second position and a second pose corresponding to a second moment in the motion trajectory; the first moment and the second moment are moments adjacent to a preset moment during the movement of the target object; Determining a time parameter based on the preset moment, the first moment, and the second moment; Determining the position of the target object based on the time parameter, the first position, and the second position; Determining the pose of the target object based on the time parameter, the first pose, and the second pose.

3. The method according to claim 1, wherein The determining the first mesh data of the target object based on the position and the pose includes: Obtaining historical mesh data; Determining the first mesh data of the target object based on the position, the pose, and the historical mesh data.

4. The method according to claim 1, wherein Before determining the second mesh data from the first mesh data based on the target scanning range of the lidar, the method further includes: Dividing the preset frame time according to a preset number of divisions to obtain the target scanning time; the preset frame time is the time required for the lidar to scan one week; Determining a scanning range sequence based on the preset number of divisions and the target scanning time; Selecting a scanning range as the target scanning range from the scanning range sequence in the sequence order.

5. The method according to claim 4, wherein The determining the point cloud data corresponding to the target object within the target scanning range based on the point cloud intensity and the intersection point coordinates includes: Whenever a target scanning range is selected from the scanning range sequence, determining the point cloud data corresponding to the target object within the selected target scanning range based on the point cloud intensity and the intersection point coordinates, until the point cloud data corresponding to the target object within each scanning range in the scanning range sequence is obtained; The method further includes: combining the point cloud data corresponding to the target objects in each scanning range in the scanning range sequence into the point cloud simulation data of the target objects.

6. The method according to claim 1, characterized in that, Determining the second grid data from the first grid data for the target scanning range based on the lidar includes: Filtering the first grid data according to the target scanning range of the lidar to obtain the second grid data in the first grid data that belongs to the target scanning range.

7. The method according to claim 1, characterized in that, Calculating the reflected point cloud intensity based on the direction vector information of the laser beam, the normal vector information of the target polygon, and the intersection coordinates includes: Determining the included angle based on the direction vector information of the laser beam and the normal vector information of the target polygon; Calculating the reflected point cloud intensity based on the included angle, the intersection coordinates, and the starting point.

8. The method according to claim 7, characterized in that The method further includes: Determining the calculation block capacity according to the preset number of beam lines and the preset number of target polygons; Determining the number of calculation blocks according to the number of laser beam lines, the number of target polygons, and the calculation block capacity; Determining the number of threads in each calculation block according to the calculation block capacity; Calculating the reflected point cloud intensity based on the direction vector information of the laser beam, the normal vector information of the target polygon, and the intersection coordinates includes: Parallel computing the direction vector information of the laser beam, the normal vector information of the target polygon, and the intersection coordinates according to the number of calculation blocks and the number of threads to obtain the reflected point cloud intensity.

9. A lidar point cloud simulation device, the device includes a memory and a processor, the memory stores a computer program, the processor includes a first processor and a second processor, and is characterized in that, When the processor executes the computer program, it implements the lidar point cloud simulation method, and the method includes: The first processor obtains the position and attitude of the target object; The second processor determines the first grid data of the target object based on the position and the attitude; The second processor determines the second grid data from the first grid data based on the target scanning range of the lidar; The second processor obtains the starting point and direction vector information of the laser beam in the target scanning range, and the normal vector information of the target polygon; calculates the distance from the starting point to the intersection point based on the starting point, the direction vector information, and the normal vector information of the target polygon; the intersection point is the intersection point of the target polygon and the laser beam; when the distance is less than the effective detection distance of the lidar, it is determined that the laser beam in the laser beam data intersects with the target polygon; calculates the intersection coordinates of the intersection point based on the starting coordinates, the distance, and the direction vector information; The second processor calculates the reflected point cloud intensity based on the direction vector information of the laser beam, the normal vector information of the target polygon, and the intersection coordinates, and the second processor determines the point cloud data corresponding to the target object in the target scanning range based on the point cloud intensity and the intersection coordinates.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1 to 8.

Images