Point cloud generation method and device, computer device and storage medium

Abstract

The application relates to a point cloud generation method and device, computer equipment and a storage medium. The method comprises the following steps: projecting an original point cloud of a target scene into a first depth map according to target spherical projection parameters; projecting a target model into a second depth map according to the target spherical projection parameters based on placement space information of the target model relative to the target scene; the second depth map comprises a target depth region corresponding to the target model obtained by projection; determining a first target pixel point in the first depth map, so that a second target pixel point in the second depth map with the same position as the first target pixel point belongs to the target depth region; performing depth value updating on the first target pixel point according to a depth value of the corresponding second target pixel point; and updating the original point cloud based on a difference point cloud obtained by inverse processing of the first target pixel point after the depth value updating, to obtain a new point cloud. The method can realize point cloud simulation with high efficiency.

Description

Technical Field

[0001] This application relates to the field of point cloud simulation technology, and in particular to a point cloud generation method, apparatus, computer equipment, and storage medium. Background Technology

[0002] With the development of computer technology, point clouds can now be obtained from the environment using measuring instruments. These point clouds reflect spatial information about the environment, and computer technology can be used to process and analyze them, finding wide application in various fields such as autonomous driving, geographic mapping, and construction. However, obtaining point clouds through measurement requires manual on-site operation, which has significant limitations. Therefore, point cloud simulation technology has emerged. Point cloud simulation is achieved by inserting a new model into the original point cloud.

[0003] Since point clouds reflect the surface information of an object initially acquired by a measuring instrument from various directions in a 3D environment, the inserted new model may occlude the original point cloud. Therefore, the occluded points in the original point cloud need to be replaced with points from the new model. This replacement process can be achieved by acquiring depth maps of both the original point cloud and the new model. Each pixel in the depth map reflects distance information; currently, depth maps are acquired using camera models.

[0004] However, the process of obtaining point cloud depth maps through camera models can only process point clouds corresponding to a certain field of view. In order to obtain a complete point cloud depth map, multiple camera models need to be set up and the point cloud depth maps obtained by multiple camera models need to be stitched together to form a complete point cloud depth map. The efficiency of obtaining depth maps is low, resulting in low efficiency of point cloud simulation. Summary of the Invention

[0005] Therefore, it is necessary to provide a point cloud generation method, apparatus, computer equipment, and computer-readable storage medium that can solve the problem of low point cloud simulation efficiency in order to address the aforementioned technical issues.

[0006] Firstly, this application provides a point cloud generation method. The method includes:

[0007] The original point cloud of the target scene is projected into a first depth map according to the target spherical projection parameters;

[0008] Based on the placement space information of the target model relative to the target scene, the target model is projected into a second depth map according to the target spherical projection parameters; the second depth map includes the target depth region corresponding to the target model obtained by projection;

[0009] Determine a first target pixel point projected from the first depth map, such that a second target pixel point in the second depth map, which has the same position as the first target pixel point, belongs to the target depth region;

[0010] The depth value of the first target pixel is updated according to the depth value of the corresponding second target pixel;

[0011] Based on the difference point cloud obtained by projecting the first target pixel after depth value update, the original point cloud is updated to obtain a new point cloud.

[0012] In one embodiment, projecting the original point cloud of the target scene into a first depth map according to the target spherical projection parameters includes:

[0013] Obtain spherical coordinate data of each data point in the original point cloud of the target scene, wherein the spherical coordinate data includes angle data and radial distance;

[0014] According to the target spherical projection parameters, the angle data of each data point is mapped to the first image coordinate data;

[0015] Determine the radial distance corresponding to each of the first image coordinate data, and generate a first depth map based on the radial distance of each of the first image coordinate data and the corresponding depth representation value.

[0016] In one embodiment, obtaining the spherical coordinate data of each data point in the original point cloud of the target scene includes:

[0017] Obtain the Cartesian coordinates of each data point in the original point cloud of the target scene;

[0018] According to the mapping relationship between the Cartesian coordinate system and the spherical coordinate system, the Cartesian coordinate data is transformed to obtain the spherical coordinate data of each data point in the original point cloud.

[0019] In one embodiment, the target spherical projection parameters include depth map side length data and angle range data; mapping the angle data to first image coordinate data according to the target spherical projection parameters includes:

[0020] For the angle data of each data point in the original point cloud, a linear mapping is performed according to the proportion of the corresponding angle data to the angle range represented by the angle range data and the side length data of the depth map to obtain the first image coordinate data corresponding to each data point.

[0021] In one embodiment, projecting the target model into a second depth map based on the placement space information of the target model relative to the target scene and according to the target spherical projection parameters includes:

[0022] Based on the placement space information of the target model relative to the target scene, determine the point ball coordinate angle data of the target model's points;

[0023] According to the target spherical projection parameters, the point ball coordinate angle data is mapped to the second image coordinate data;

[0024] Obtain the depth value corresponding to each second image coordinate data, and generate a second depth map based on the second image coordinate data and the corresponding depth value.

[0025] In one embodiment, obtaining the depth value corresponding to each second image coordinate data and generating a second depth map based on the second image coordinate data and the corresponding depth value includes:

[0026] Determine the depth buffer to use when shading the projected target model;

[0027] Read the depth value corresponding to each second image coordinate data from the depth buffer;

[0028] A second depth map is generated based on the coordinate data of each second image and its corresponding depth value.

[0029] In one embodiment, updating the depth value of the first target pixel according to the depth value of the corresponding second target pixel includes:

[0030] Compare the depth value of the first target pixel with the corresponding depth value of the second target pixel;

[0031] If the depth value of the first target pixel is greater than the depth value of the corresponding second target pixel, the depth value of the first target pixel is updated to the depth value of the corresponding second target pixel.

[0032] Secondly, this application also provides a point cloud generation apparatus. The apparatus includes:

[0033] The point cloud spherical projection module is used to project the original point cloud of the target scene into a first depth map according to the target spherical projection parameters;

[0034] The target model spherical projection module is used to project the target model into a second depth map based on the placement space information of the target model relative to the target scene and according to the target spherical projection parameters; the second depth map includes the target depth region corresponding to the target model obtained by projection;

[0035] The depth value update module is used to determine the first target pixel point projected from the first depth map, such that the second target pixel point in the second depth map with the same position as the first target pixel point belongs to the target depth region; and to update the depth value of the first target pixel point according to the depth value of the corresponding second target pixel point.

[0036] The point cloud update module is used to update the original point cloud based on the difference point cloud obtained by inverse projection processing of the first target pixel point after depth value update, so as to obtain a new point cloud.

[0037] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps:

[0038] The original point cloud of the target scene is projected into a first depth map according to the target spherical projection parameters;

[0039] Based on the placement space information of the target model relative to the target scene, the target model is projected into a second depth map according to the target spherical projection parameters; the second depth map includes the target depth region corresponding to the target model obtained by projection;

[0040] Determine a first target pixel point projected from the first depth map, such that a second target pixel point in the second depth map, which has the same position as the first target pixel point, belongs to the target depth region;

[0041] The depth value of the first target pixel is updated according to the depth value of the corresponding second target pixel;

[0042] Based on the difference point cloud obtained by projecting the first target pixel after depth value update, the original point cloud is updated to obtain a new point cloud.

[0043] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps:

[0044] The original point cloud of the target scene is projected into a first depth map according to the target spherical projection parameters;

[0045] Based on the placement space information of the target model relative to the target scene, the target model is projected into a second depth map according to the target spherical projection parameters; the second depth map includes the target depth region corresponding to the target model obtained by projection;

[0046] Determine a first target pixel point projected from the first depth map, such that a second target pixel point in the second depth map, which has the same position as the first target pixel point, belongs to the target depth region;

[0047] The depth value of the first target pixel is updated according to the depth value of the corresponding second target pixel;

[0048] Based on the difference point cloud obtained by projecting the first target pixel after depth value update, the original point cloud is updated to obtain a new point cloud.

[0049] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, performs the following steps:

[0050] The original point cloud of the target scene is projected into a first depth map according to the target spherical projection parameters;

[0051] Based on the placement space information of the target model relative to the target scene, the target model is projected into a second depth map according to the target spherical projection parameters; the second depth map includes the target depth region corresponding to the target model obtained by projection;

[0052] Determine a first target pixel point projected from the first depth map, such that a second target pixel point in the second depth map, which has the same position as the first target pixel point, belongs to the target depth region;

[0053] The depth value of the first target pixel is updated according to the depth value of the corresponding second target pixel;

[0054] Based on the difference point cloud obtained by projecting the first target pixel after depth value update, the original point cloud is updated to obtain a new point cloud.

[0055] The aforementioned point cloud generation method, apparatus, computer equipment, and storage medium perform spherical projection on the original point cloud of the target scene and the points of the target model, respectively. The projection process uses the obtained spherical projection parameters of the original point cloud, eliminating the need for additional sensor parameters. Furthermore, the projection surface formed by the spherical projection has a wider field of view. Moreover, based on the second target pixel in the target depth region of the target model, the depth values ​​of the corresponding first target pixel are updated. The first target pixel with the updated depth values ​​is then subjected to inverse projection processing to obtain a difference point cloud. The original point cloud is updated based on the difference point cloud to generate a new point cloud. The generated new point cloud only undergoes the necessary depth value updates compared to the original point cloud, retaining the original point cloud without depth value updates. The resulting new point cloud has a wider field of view, requires fewer additional parameters, and involves a relatively smaller amount of updated data. Compared to methods using camera models, this method is more efficient in acquiring depth maps, and consequently, more efficient in point cloud simulation. Attached Figure Description

[0056] Figure 1 This is an application environment diagram of a point cloud generation method in one embodiment;

[0057] Figure 2This is a flowchart illustrating a point cloud generation method in one embodiment;

[0058] Figure 3 This is a flowchart illustrating the step of projecting the original point cloud of a target scene into a first depth map according to the target spherical projection parameters in one embodiment.

[0059] Figure 4 This is a flowchart illustrating the step of projecting a target model into a second depth map based on the placement space information of the target model relative to the target scene, according to the target spherical projection parameters, in one embodiment.

[0060] Figure 5 This is a flowchart illustrating the steps of obtaining the depth value corresponding to each second image coordinate data and generating a second depth map based on the second image coordinate data and the corresponding depth value in one embodiment.

[0061] Figure 6 This is the original point cloud distribution map in an autonomous driving scenario;

[0062] Figure 7 The first depth map is obtained by projecting each data point of the original point cloud onto a spherical surface.

[0063] Figure 8 The second depth map is obtained by projecting the points of the car model onto a spherical surface;

[0064] Figure 9 To obtain a new point cloud image by adding a car model to an autonomous driving scenario;

[0065] Figure 10 This is a structural block diagram of a point cloud generation device in one embodiment;

[0066] Figure 11 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0068] The point cloud generation method provided in this application embodiment can be applied to, for example, Figure 1The application environment is shown. The point cloud generation method is primarily executed on computer device 102. Specifically, computer device 102 acquires the original point cloud and target model of the target scene, and performs a series of processes based on the original point cloud and target model to obtain a new point cloud. The original point cloud and target model can be stored in data storage system 104, which can be located on computer device 102 or in other locations accessible to computer device 102, such as servers or other devices connected to computer device 102. The original point cloud can be collected from the target scene 106 by point cloud sensor 108. The target scene can be any scene that can utilize point cloud technology, such as an autonomous driving scene, a geographic mapping scene, or a construction scene. The target model can be an object model or a map model. The object model can be a 3D model composed of any physical or virtual object, and the map model can be a 3D model composed of any virtual or measured map. Computer device 102 can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, or smart vehicle devices.

[0069] In one embodiment, such as Figure 2 As shown, a point cloud generation method is provided, which can be applied to... Figure 1 Taking computer device 102 as an example, the following steps are included:

[0070] Step 202: Project the original point cloud of the target scene into a first depth map according to the target spherical projection parameters.

[0071] The target scene is a real-world scene from which point clouds can be collected, specifically such as an autonomous driving scenario, a geographic mapping scenario, or a construction scenario. The raw point cloud is a point cloud collected at the target scene using point cloud sensors, consisting of multiple data points, each of which identifies a specific location within the target scene.

[0072] In this embodiment, projection refers to projecting a three-dimensional point cloud onto a plane. Target spherical projection parameters refer to the parameters required to project the point cloud onto the plane in a spherical coordinate system. Specifically, these may include projection parameters representing the mapping relationship before and after data projection, and scale parameters representing the depth map size.

[0073] Depth maps can be in image format or non-image format. Image format depth maps record the position and depth values ​​of 3D data projected onto a 2D plane, where the pixel value of each pixel in the depth map represents its depth. Non-image format depth maps can be matrices, recording the data after 3D data is projected onto a 2D plane, and the correspondence between the 3D data before and after projection and the data on the 2D plane; the 2D plane represents the depth map and can be viewed as a matrix, with each number in the matrix representing a pixel in the depth map. The first depth map is obtained by projecting the original point cloud, and is distinguished from the second depth map described below.

[0074] Specifically, in one embodiment, the computer device can read the original point cloud collected from the target scene and obtain the target spherical projection parameters. In the spherical coordinate system, each data point in the original point cloud is projected onto the plane according to the target spherical projection parameters. The computer device determines the position and depth value of each data point after it is projected onto the plane, thereby generating a first depth map based on the position and depth values.

[0075] In one embodiment, the target spherical projection parameters may include projection parameters representing the mapping relationship before and after data projection, and scale parameters representing the depth map size. The computer device projects each data point in the original point cloud onto a plane according to the target spherical projection parameters, specifically including: the computer device projects each data point in the original point cloud onto a first depth map according to the target spherical projection parameters, such that the positions of the data points before projection and the pixels after projection conform to the mapping relationship represented by the projection parameters in the target spherical projection parameters, and the size of the first depth map conforms to the depth map size represented by the scale parameters in the target spherical projection parameters.

[0076] Step 204: Based on the placement space information of the target model relative to the target scene, project the target model into a second depth map according to the target spherical projection parameters; the second depth map includes the target depth region corresponding to the target model obtained by projection.

[0077] The target model is a three-dimensional model, which can be a simulation model generated by collecting point clouds of objects in the real world, a simulation model directly generated using 3D simulation technology, or a 3D model of a virtual avatar. The target model can be an object model or a map model. The simulated object in the object model can be a living or non-living object, a naturally existing object or a man-made object. The target model consists of several planar elements, which are connected by edges and vertices, with vertices shared by multiple planar elements. Planar elements can be triangles or quadrilaterals.

[0078] Placement spatial information refers to the spatial information of the target model in the target scene assuming that the target model is placed in the target scene. The placement spatial information may include, but is not limited to, the spatial position and orientation of the target model relative to the target scene.

[0079] The target depth region in the second depth map is the area that records the depth values ​​corresponding to the points of the target model after projection. Specifically, when the points of the target model are projected onto the second depth map, any point of the target model participating in the projection will be projected onto a corresponding pixel in the second depth map. The depth value of this pixel is greater than zero, while other pixels in the second depth map that are not projected from the points of the target model have a zero value. In this case, the region formed by the pixels with non-zero depth values ​​in the second depth map is the target depth region. The points of the target model can be vertices of the target model or points obtained by uniformly sampling at sampling intervals within the planar elements of the target model.

[0080] Specifically, in one embodiment, the computer device can acquire placement space information set for the target model, and then, based on this placement space information, project the points of the target model onto a plane using the same target spherical projection parameters as when the first depth map was obtained, to obtain a second depth map. The second depth map has the same dimensions as the first depth map.

[0081] Furthermore, in one embodiment, the target spherical projection parameters may include projection parameters representing the mapping relationship before and after data projection, and scale parameters representing the depth map size. Specifically, the computer device may project each point of the target model participating in the projection onto a plane according to the target spherical projection parameters, such that the positions of the points before projection and the pixels after projection conform to the mapping relationship represented by the projection parameters in the target spherical projection parameters, and the size of the second depth map conforms to the depth map size represented by the scale parameters in the target spherical projection parameters.

[0082] Step 206: Determine the first target pixel point projected from the first depth map, such that the second target pixel point in the second depth map, which has the same position as the first target pixel point, belongs to the target depth region.

[0083] The first depth map and the second depth map are the same size, and each pixel in the first depth map has a corresponding pixel in the second depth map. Both the first target pixel and the second target pixel are projected pixels; for each first target pixel in the first depth map, there is a corresponding second target pixel at the same position in the second depth map; and the second target pixel corresponding to each first target pixel falls within the target depth region.

[0084] Specifically, the computer device can traverse each data point in the original point cloud and determine whether the pixel at the same position in the second depth map corresponding to the pixel of the traversed data point projected onto the first depth map exists in the target depth region; if it exists, the pixel obtained by projecting the traversed data point is the first target pixel; if it does not exist, the pixel obtained by projecting the traversed data point is not the first target pixel.

[0085] In one embodiment, the computer device may traverse the pixels in the first depth map and determine whether the traversed pixels are projected pixels. If they are projected pixels, the device further determines whether the pixels at the same position in the second depth map corresponding to the traversed pixels are in the target depth region. If they exist, the traversed pixels are the first target pixels. If they do not exist, or if the traversed pixels are not projected pixels, the traversed pixels are not the first target pixels.

[0086] In one embodiment, the computer device may traverse the pixels in the second depth map and determine whether the traversed pixels are in the target depth region. If they are in the target depth region, the device may determine whether the pixel at the same position in the first depth map corresponding to the traversed pixels is a projected pixel. If it is a projected pixel, the traversed pixel is the second target pixel, and the pixel corresponding to the traversed pixel in the first depth map is the first target pixel. If it is not a projected pixel, or the traversed pixel is not in the target depth region, the traversed pixel is not the second target pixel, and the pixel corresponding to the traversed pixel in the first depth map is not the first target pixel.

[0087] Step 208: Update the depth value of the first target pixel according to the depth value of the corresponding second target pixel.

[0088] The original point cloud of the target scene is collected by a point cloud sensor. It is expected that a target model will be added to the target scene. At this time, the target model and the original point cloud may form an occlusion relationship relative to the point cloud sensor. When a new point cloud is formed, the occluded data points in the original point cloud need to be replaced with points of the target model.

[0089] For each first target pixel, there exists a corresponding data point in the original point cloud. Correspondingly, for each second target pixel, there exists a corresponding point on the target model. The computer device can determine whether there is an occlusion relationship between the corresponding data point and the corresponding point of the second target pixel for the first target pixel determined in step 206. If there is no occlusion relationship, there is no need to update the depth value of the first target pixel; if there is an occlusion relationship and the point is occluded by the data point, there is also no need to update the depth value of the first target pixel; if there is an occlusion relationship and the data point is occluded by the point, the depth value of the second target pixel at the same position as the first target pixel in the second depth map is obtained, and the depth value of the first target pixel is updated in the first depth map according to the obtained depth value.

[0090] In one embodiment, the computer device can traverse each data point in the original point cloud and determine whether the pixel projected onto the first depth map by the traversed data point is a first target pixel. If it is not a first target pixel, the traversal continues to the next data point. If it is a first target pixel, and the data point corresponding to the first target pixel is occluded by the point corresponding to the second target pixel, the depth value of the first target pixel in the first depth map is updated according to the depth value of the corresponding second target pixel.

[0091] In one embodiment, the computer device can traverse the pixels in the first depth map and determine whether the traversed pixel is a first target pixel. If it is not the first target pixel, the traversal continues to the next pixel. If it is the first target pixel, and the data point corresponding to the first target pixel is occluded by the point corresponding to the second target pixel, the depth value of the first target pixel in the first depth map is updated according to the depth value of the corresponding second target pixel.

[0092] In one embodiment, the computer device can traverse the pixels in the second depth map and determine whether the traversed pixel is a second target pixel. If it is not a second target pixel, the traversal continues to the next pixel. If it is a second target pixel, and the data point corresponding to the first target pixel is occluded by the point corresponding to the second target pixel, the depth value of the first target pixel in the first depth map is updated according to the depth value of the corresponding second target pixel.

[0093] Step 210: Based on the difference point cloud obtained by projecting the first target pixel after the depth value update, update the original point cloud to obtain a new point cloud.

[0094] Inverse projection processing is the reverse of projection in step 202. Through projection, data points in the original point cloud can be mapped to pixels in the first depth map; through inverse projection processing, pixels in the first depth map can be mapped to data points in the original point cloud. During projection, one or more data points can be mapped to a single pixel; during inverse projection processing, a pixel can be directly mapped to a data point using mapping relationships, or a pixel can be mapped to multiple data points by referencing information recorded during projection. The difference point cloud refers to the point cloud composed of data points obtained by performing inverse projection processing on the first target pixel after depth value updates.

[0095] In one embodiment, the computer device may perform inverse projection processing on the first target pixel that has been updated with depth values ​​to obtain a difference point cloud. Further, the computer device may update the original point cloud based on the difference point cloud to obtain a new point cloud.

[0096] In one embodiment, the computer device can map the pixel position and depth value of the first target pixel to coordinate data of data points based on the target spherical projection and the projection formula, thereby obtaining a difference point cloud. In other embodiments, the computer device can also refer to the information of the data points recorded when projected onto the first target pixel, and update the corresponding data points in the original point cloud before projection onto the first target pixel based on the pixel position and depth value of the first target pixel, to obtain a difference point cloud.

[0097] In one embodiment, after performing inverse projection processing on each first target pixel to obtain data points in the difference point cloud, the computer device uses these data points to replace the data points projected onto the first target pixel in the original point cloud, thereby updating the original point cloud based on the difference point cloud to obtain a new point cloud. In other embodiments, the computer device may also perform inverse projection processing on all first target pixels to obtain a complete difference point cloud, and then update the complete original point cloud based on the complete difference point cloud to obtain a new point cloud. During the update process, the data points obtained by performing inverse projection processing on each first target pixel are used to replace the data points projected onto the first target pixel in the original point cloud.

[0098] In the above point cloud generation method, the original point cloud of the target scene and the points of the target model are spherically projected. The projection process uses the obtained spherical projection parameters of the original point cloud, without the need to obtain additional sensor parameters. At the same time, the projection surface formed by the spherical projection has a wider field of view. Furthermore, the depth value of the corresponding first target pixel is updated according to the second target pixel of the target depth region of the target model, and the first target pixel with the updated depth value is subjected to inverse projection processing to obtain the difference point cloud. The original point cloud is updated based on the difference point cloud to generate a new point cloud. The generated new point cloud only has the necessary depth value update compared to the original point cloud, and the original point cloud without the depth value update is retained. The final new point cloud has a wider field of view, and fewer additional parameters are required in the process, and the amount of data updated is relatively small. Compared with the method using a camera model, the efficiency of obtaining depth maps is higher, and the efficiency of point cloud simulation is also higher.

[0099] In one embodiment, such as Figure 3 As shown, step 202 includes the following steps 302 to 306:

[0100] Step 302: Obtain the spherical coordinate data of each data point in the original point cloud of the target scene. The spherical coordinate data includes angle data and radial distance.

[0101] The spherical coordinate data of the data points in the original point cloud refers to the coordinates of those data points in a spherical coordinate system, including angle data and radial distance. The angle data includes azimuth and pitch angles. The azimuth angle is the angle between the ray pointing from the origin to the data point in the horizontal direction and the horizontal reference line. The pitch angle is the angle between the ray pointing from the origin to the data point in the vertical direction and the vertical reference line. The radial distance is the straight-line distance from the origin to the data point in the spherical coordinate system. When the vertical reference line is a ray extending vertically upwards from the origin, the pitch angle is the zenith angle.

[0102] Step 304: Map the angle data of each data point to the first image coordinate data according to the target spherical projection parameters.

[0103] The target spherical projection parameters include projection parameters representing the mapping relationship before and after data projection, and scale parameters representing the depth map size. In the spherical coordinate system, with the origin as the observation point, the angle data of each data point in the spherical coordinate system determines the position of the data point seen from the observation point, while the radial distance determines the straight-line distance between the observation point and the data point.

[0104] The scale parameter in the target spherical projection parameters determines the size of the projected depth map, which in turn determines the range of the projected image coordinate data. The projection parameter in the target spherical projection parameters characterizes the mapping relationship between the data before and after projection. Therefore, based on the projection parameter and scale parameter in the target spherical projection parameters, the angle data of each data point can be mapped to the first image coordinate data.

[0105] Step 306: Determine the radial distance corresponding to each first image coordinate data, and generate a first depth map based on each first image coordinate data and the corresponding radial distance representing the depth.

[0106] Specifically, the spherical coordinate data of a data point includes angle data and radial distance. In step 304, the spherical coordinate data of this data point is mapped to first image coordinate data. Then, the radial distance of this data point can be used as the corresponding radial distance of the first image coordinate data. In the spherical coordinate system, the radial distance represents the straight-line distance between the data point and the observation point. This is consistent with the positional relationship between the point cloud sensor and the data point in the target scene. Therefore, this radial distance can be used to represent the depth value of the data point.

[0107] In one embodiment, the computer device can determine the pixel points projected onto the first depth map based on the first image coordinate data, thereby determining the pixel value of the pixel point using the corresponding radial distance of the first image coordinate data. This pixel value represents the depth value. Other pixels in the first depth map that are not projected are assigned a default pixel value, which can be zero, indicating that the pixel point is not a projection of the data points.

[0108] In this embodiment, the characteristics of spherical coordinates and point clouds are cleverly utilized. When the origin is used as the observation point in the spherical coordinate system, the angle data of the data points can represent the position of the data points as seen from the observation point. When projected onto a plane, the angle data can be directly used to determine the image coordinates. The radial distance of the data points can be regarded as the distance from the data points to the observation point, corresponding to the meaning of the depth value in the point cloud, and can be directly used to represent the depth value. In this way, the position of the data points in the original point cloud can be accurately represented by the two-dimensional first depth map, providing conditions for efficient and accurate point cloud simulation.

[0109] In one embodiment, step 302 includes: obtaining the Cartesian coordinate data of each data point in the original point cloud of the target scene; and performing coordinate system transformation on the Cartesian coordinate data according to the mapping relationship between the Cartesian coordinate system and the spherical coordinate system to obtain the spherical coordinate data of each data point in the original point cloud.

[0110] Cartesian coordinate data refers to coordinate data in the Cartesian coordinate system, a three-dimensional rectangular coordinate system composed of three mutually perpendicular number axes intersecting at the origin: the x-axis, y-axis, and z-axis. Correspondingly, Cartesian coordinate data consists of x-axis (x), y-axis (y), and z-axis (z). The spherical coordinate system, on the other hand, is a three-dimensional spatial coordinate system with the origin as the reference point. The coordinates of data points in the spherical coordinate system are expressed in terms of azimuth angle. It is represented by the zenith angle (θ) and radial distance (r).

[0111] By using the mapping relationship between the Cartesian coordinate system and the spherical coordinate system, the coordinate data of data points in the Cartesian coordinate system can be mapped to the coordinate data in the spherical coordinate system. Specifically, the coordinate data of data points in the spherical coordinate system can be calculated using the following formulas (1), (2), and (3):

[0112]

[0113]

[0114]

[0115] in, It represents the azimuth angle of a data point in a spherical coordinate system, which is equal to the arctangent of the ratio between the vertical coordinate (y) and the horizontal coordinate (x) of the data point in a Cartesian coordinate system.

[0116] θ represents the zenith angle of the data point in the spherical coordinate system. It is equal to the arctangent of the ratio of the calculated value between the horizontal axis coordinate (x) and the vertical axis coordinate (y) of the data point in the Cartesian coordinate system to the vertical axis coordinate (z). This calculated value is the square root of the sum of the squares of the horizontal axis coordinate (x) and the vertical axis coordinate (y) of the data point in the Cartesian coordinate system.

[0117] r represents the radial distance of the data point in the spherical coordinate system, which is equal to the square root of the sum of the squares of the x-axis (x), y-axis (y), and z-axis (z) coordinates of the data point in the Cartesian coordinate system.

[0118] Specifically, in one embodiment, the computer reads the Cartesian coordinate data of each data point in the original point cloud of the target scene, and maps the Cartesian coordinate data of each data point to the spherical coordinate data of each data point according to the mapping relationship between the Cartesian coordinate system and the spherical coordinate system, that is, the mapping relationship represented by the above formulas (1) to (3), so as to obtain the azimuth angle, zenith angle and radial distance of each data point in the original point cloud in the spherical coordinate system.

[0119] In this embodiment, the data points in the original point cloud are represented using coordinate data in the Cartesian coordinate system, which corresponds to the real space and facilitates understanding and computation. Utilizing the fixed mapping relationship between the Cartesian and spherical coordinate systems, a direct conversion yields spherical coordinate data, which can then be used for direct projection to obtain the first depth map. The entire process is simple and efficient, providing conditions for efficient and accurate point cloud simulation.

[0120] In one embodiment, the target spherical projection parameters include depth map side length data and angle range data. Step 304 includes: for the angle data of each data point in the original point cloud, performing linear mapping according to the angle ratio of the corresponding angle data to the angle range represented by the angle range data and the depth map side length data to obtain the first image coordinate data corresponding to each data point.

[0121] The target spherical projection parameters include scale parameters, which include depth map side length data. The depth map side length data includes a first side length and a second side length, which are respectively taken as either the depth map width (W) or the depth map height (H). Together, the first and second side lengths determine the depth map size. The target spherical projection parameters also include projection parameters, which include angle range data. The angle range data includes at least zenith angle range data and may also include azimuth angle range data. The angle range data represents the range to which the angle data of a data point belongs, and can be represented by the boundary angles of the corresponding angle range or by the absolute value of the difference between the boundary angles of the corresponding angle range.

[0122] Specifically, the computer device can acquire the angle data of each data point and the angle range to which that angle data belongs. Based on the angle range data, it determines the angle range to which the angle data belongs, and then calculates the proportion of the angle data to that angle range. This proportion is then used to perform a linear mapping with the depth map side length data to determine the first image coordinate data corresponding to each data point. The proportion of this first image coordinate data to the corresponding depth map side length data is linearly correlated with the proportion of the data point's angle data to that angle range.

[0123] Further, in one embodiment, the computer device can acquire the azimuth angle and azimuth angle range of each data point, calculate the proportion of the azimuth angle to the azimuth angle range, and then perform a linear mapping based on the proportion and a first side length to obtain the first image coordinate value of the data point mapped to the first depth map relative to the first side length. In one embodiment, the computer device can acquire the zenith angle and zenith angle range of each data point, calculate the proportion of the zenith angle to the zenith angle range, and then perform a linear mapping based on the proportion and a second side length to obtain the second image coordinate value of the data point mapped to the zenith angle to the first depth map relative to the second side length. The first image coordinate value and the second image coordinate value constitute the first image coordinate data corresponding to each data point.

[0124] In one embodiment, the azimuth range data can be the boundary angles of the azimuth domain, and the corresponding azimuth range can be the absolute value of the difference between the boundary angles of the azimuth domain, such as 2π. In another embodiment, the azimuth range data can be the maximum and minimum azimuth angles of the original point cloud, and the corresponding azimuth range is the absolute value of the difference between the maximum and minimum azimuth angles.

[0125] In one embodiment, the zenith angle range data can be the boundary angle of the zenith angle domain, and the corresponding zenith angle range can be the absolute value of the boundary angle of the zenith angle domain, such as π. In another embodiment, the zenith angle range data can be the maximum and minimum zenith angles of the original point cloud, and the corresponding zenith angle range is the absolute value of the difference between the maximum and minimum zenith angles.

[0126] In one embodiment, the first side length is taken as the width of the depth map, and the second side length is taken as the height of the depth map. The coordinate values ​​of the first image and the second image can be calculated using the following formulas (4) and (5) respectively:

[0127]

[0128]

[0129] Where u represents the first image coordinate value corresponding to the azimuth projection of the data point onto the first depth map, which is equal to the product of the ratio of the azimuth calculation of the data point to the azimuth range and the first side length. The azimuth calculation is the difference between the azimuth angle and the minimum azimuth angle.

[0130] v represents the second image coordinate value corresponding to the zenith angle projection of the data point onto the second depth map. It is equal to the negative value of the product of the ratio of the zenith angle calculation of the data point to the zenith angle range and the second side length. The zenith angle calculation is the difference between the zenith angle and the minimum zenith angle.

[0131] Specifically, in one embodiment, the computer device can acquire the azimuth angle, minimum azimuth angle, azimuth angle range, zenith angle, minimum zenith angle and zenith angle range of each data point, and map the azimuth angle and zenith angle of each data point to the first image coordinate value and the second image coordinate value respectively according to the mapping relationship represented by formulas (4) and (5). The first image coordinate value and the second image coordinate value constitute the first image coordinate data.

[0132] In formulas (4) and (5) above, the origin of the spherical coordinate system is mapped to the upper left corner of the pixel in the first depth map. When the origin is mapped to a pixel in a different position in the first depth map, other linear formulas can be used.

[0133] In one embodiment, the first image coordinate values ​​can be calculated using the following formulas (6) and (7):

[0134]

[0135]

[0136] Where u represents the first image coordinate value corresponding to the azimuth projection of the data point onto the first depth map. It is the azimuth angle, which ranges from 0 to 2π, or from -π to π, depending on the reference point. W is the width of the first depth map.

[0137] v represents the θ of the data point fov Projecting the first image coordinates onto the first depth map. θ fov fov is the angle between the ray pointing from the origin to the data point in the vertical direction and the horizontal plane passing through the origin in the spherical coordinate system; it is the pitch angle with the horizontal plane as the reference point. down The maximum lower field of view angle is the maximum angle between the ray pointing from the origin to the data points distributed in the lower hemisphere of the spherical coordinate system in the vertical direction and the horizontal plane passing through the origin; fov up θ is the maximum upward field of view angle, which is the maximum angle between the ray pointing from the origin to the data points distributed in the upper hemisphere of the spherical coordinate system and the horizontal plane passing through the origin in the vertical direction. fov The range of values ​​for fov is down to fov up H represents the height of the first depth map.

[0138] In this embodiment, the angle data accounts for the proportion of the angle range represented by the angle range data, and a first depth map is generated by linear mapping based on the angle proportion and the depth map side length data. The calculation is simple and efficient, providing conditions for efficient and accurate point cloud simulation.

[0139] In one embodiment, such as Figure 4 As shown, step 204 includes the following steps 402 to 406:

[0140] Step 402: Based on the placement space information of the target model relative to the target scene, determine the point-ball coordinate angle data of the target model's points.

[0141] Among them, the point spherical coordinate angle data is the angle data of the point in the spherical coordinate system. This spherical coordinate system is the same as the spherical coordinate system in which each data point in the original point cloud is projected from the spherical coordinate system to the first depth map. The point spherical coordinate angle data also includes azimuth and pitch angles, and the pitch angle includes the zenith angle.

[0142] Specifically, in one embodiment, the computer device reads the target model, determines the placement space information of the target model relative to the target scene, and determines the point-spherical coordinate angle data of the points of the target model in the spherical coordinate system based on the placement space information.

[0143] In one embodiment, a computer device may acquire Cartesian coordinate data of points of a target model in a Cartesian coordinate system, which is the same Cartesian coordinate system in which each data point in the original point cloud is projected from the Cartesian coordinate system onto the first depth map.

[0144] Specifically, in one embodiment, the computer device reads the target model, determines the placement space information of the target model relative to the target scene, and based on the placement space information, determines the Cartesian coordinate data (x', y', z') of the points of the target model in the Cartesian coordinate system. The Cartesian coordinate data of the points of the target model in the Cartesian coordinate system can be mapped to the spherical coordinate data in the spherical coordinate system. Specifically, the following formulas (8) and (9) can be used to calculate the spherical coordinate data of the points of the target model in the spherical coordinate system:

[0145]

[0146]

[0147] in, The azimuth angle of a point in the target model in spherical coordinates is equal to the arctangent of the ratio between the vertical coordinate (y') and the horizontal coordinate (x') of that point in Cartesian coordinates.

[0148] θ' represents the zenith angle of a point in the target model in spherical coordinates. It is equal to the arctangent of the ratio of the calculated value between the x-axis coordinate (x') and y-axis coordinate (y') of the point in Cartesian coordinates to the vertical coordinate (z'). This calculated value is the square root of the sum of the squares of the x-axis coordinate (x') and y-axis coordinate (y') of the point in Cartesian coordinates.

[0149] Step 404: Map the point ball coordinate angle data to the second image coordinate data according to the target spherical projection parameters.

[0150] Specifically, in one embodiment, the computer device acquires the point-sphere coordinate angle data of the target model participating in the projection, and maps the point-sphere coordinate angle data into second image coordinate data according to the projection parameters and scale parameters in the target spherical projection parameters.

[0151] In one embodiment, the computer device acquires the azimuth angle of the point sphere coordinates of the target model participating in the projection, calculates the proportion of this azimuth angle to the range of azimuth angles of each data point in the original point cloud, and then performs a linear mapping based on this proportion and a first side length to obtain a third image coordinate value of the point sphere coordinates mapped to the second depth map relative to the first side length. In another embodiment, the computer device can acquire the zenith angle of the point sphere coordinates of the target model participating in the projection, calculate the proportion of this zenith angle to the range of zenith angles of each data point in the original point cloud, and then perform a linear mapping based on this proportion and a second side length to obtain a fourth image coordinate value of the point sphere coordinates mapped to the second depth map relative to the second side length. The third and fourth image coordinate values ​​constitute the second image coordinate data corresponding to the point participating in the projection.

[0152] In one embodiment, the computer device obtains the point sphere coordinates and azimuth angles of the target model participating in the projection. The zenith angle (θ') of the point sphere coordinates can be calculated using the following formulas (10) and (11) to calculate the coordinate values ​​of the third and fourth images respectively:

[0153]

[0154]

[0155] Where u' represents the point sphere coordinate azimuth angle of the target model participating in the projection, projected onto the third image coordinate value corresponding to the second depth map, which is equal to the product of the ratio of the point sphere coordinate azimuth angle calculation to the azimuth angle range and the first side length. The point sphere coordinate azimuth angle calculation is the difference between the point sphere coordinate azimuth angle and the minimum azimuth angle.

[0156] v' represents the zenith angle of the point in the target model participating in the projection, projected onto the third image coordinate value corresponding to the second depth map. It is equal to the negative value of the product of the ratio of the zenith angle operation to the zenith angle range and the second side length. The zenith angle operation is the difference between the zenith angle and the minimum zenith angle.

[0157] In the above formulas (10) and (11), the origin of the spherical coordinate system is mapped to the pixel point in the second depth map, which is located at the upper left corner of the second depth map.

[0158] Step 406: Obtain the depth value corresponding to each second image coordinate data, and generate a second depth map according to the second image coordinate data and the corresponding depth value.

[0159] When the points of the target model involved in the projection are projected from the 3D coordinate system to the second depth map, there may be occlusion relationships between the points relative to the origin of the 3D coordinate system. When obtaining the depth value corresponding to the second image coordinate data, it is necessary to determine the occlusion of the points before projection corresponding to the second image coordinate data before obtaining the depth value corresponding to the second image coordinate data. In one embodiment, a depth buffering technique can also be used on the plane obtained by projection of the points of the target model involved in the projection to directly obtain the depth value corresponding to the second image coordinate data.

[0160] Specifically, in one embodiment, the computer device acquires the second image coordinate data of the points of the target model participating in the projection, and acquires the depth value corresponding to the second image coordinate data, and generates a second depth map based on each second image coordinate data and the corresponding depth value.

[0161] In this embodiment, the spherical projection parameters of each data point of the original point cloud when projected onto the first depth map are used to generate a second depth map by spherical projection of the points of the target model. This ensures that the first and second depth maps have the same size and that the pixels on the first and second depth maps correspond one-to-one, providing a basis for subsequent depth value updates of the first depth map and thus creating conditions for efficient and accurate point cloud simulation.

[0162] In one embodiment, such as Figure 5 As shown, step 406 includes the following steps 502 to 506:

[0163] Step 502: Determine the depth buffer to be used when shading the projected target model.

[0164] Shading is a process of calculating the lighting effects of a 3D model and filling its planar elements with appropriate colors. The shading process begins by determining the view plane of the 3D model and then drawing it based on that plane. Relative to the view plane, the computer determines whether to shade the 3D model by performing depth testing on the pixels within the model. A depth buffer is set on the view plane, and the positions of pixels in the depth buffer are correlated one-to-one with the positions of pixels in the 3D model. The initial values ​​of the pixels in the depth buffer are set to the deepest possible depth. The depth testing process involves comparing the depth values ​​of each pixel in the 3D model with the corresponding depth values ​​in the depth buffer. This process is repeated for each pixel in the 3D model, comparing the depth values ​​of the traversed pixels with the corresponding depth values ​​in the depth buffer.

[0165] If the depth value of a traversed pixel is less than the depth value of the corresponding pixel in the depth buffer, then the depth value of the corresponding pixel in the depth buffer is replaced with the depth value of the traversed pixel; if the depth value of a traversed pixel is greater than or equal to the depth value of the corresponding pixel in the depth buffer, then the depth value of the corresponding pixel in the depth buffer remains unchanged. The 3D model is then shaded based on the pixel depth values ​​in the depth buffer. Accordingly, by retrieving the depth buffer used in the 3D model shading process, the depth values ​​of the pixels in the 3D model can be determined.

[0166] Specifically, in one embodiment, the computer device determines the planar image where the second image coordinate data is located, the pixel values ​​of each pixel on the planar image are default values, performs color processing on the planar image, and determines the depth buffer used in the color processing.

[0167] Step 504: Read the depth value corresponding to each second image coordinate data from the depth buffer.

[0168] The depth buffer is determined in step 502, and the depth buffer contains pixels that correspond one-to-one with the pixels of the planar image.

[0169] Specifically, in one embodiment, the computer reads the corresponding depth value from the depth buffer for each second image coordinate data.

[0170] Step 506: Generate a second depth map based on each second image coordinate data and its corresponding depth value.

[0171] Specifically, the computer device acquires the second image coordinate data obtained by projecting the target model, and determines a corresponding depth value for each second image coordinate data. This depth value is the pixel value of the pixel corresponding to the second image coordinate data. Other pixels in the second depth map that are not projected are assigned a default pixel value, which can be zero, indicating that the pixel is not obtained by projecting points from the target model.

[0172] In this embodiment, the depth buffer generated during the coloring process is cleverly utilized to color the two-dimensional plane after the points of the target model participating in the projection are projected. This can efficiently obtain the depth value corresponding to the second image coordinate data, thereby obtaining the second depth map, which provides conditions for efficient and accurate point cloud simulation.

[0173] In one embodiment, updating the depth value of the first target pixel according to the depth value of the corresponding second target pixel includes:

[0174] The depth value of the first target pixel is compared with the depth value of the corresponding second target pixel; if the depth value of the first target pixel is greater than the depth value of the corresponding second target pixel, the depth value of the first target pixel is updated to the depth value of the corresponding second target pixel.

[0175] Specifically, in one embodiment, the computer device determines a first target pixel and a second target pixel, traverses the first target pixel, and determines whether the depth value of the traversed first target pixel is greater than the depth value of the corresponding second target pixel; if the depth value of the traversed first target pixel is greater than the depth value of the corresponding second target pixel, the depth value of the traversed first target pixel is changed to the depth value of the corresponding second target pixel; if the depth value of the traversed first target pixel is not greater than the depth value of the corresponding second target pixel, the depth value of the traversed first target pixel remains unchanged.

[0176] In one embodiment, the computer device determines a first target pixel and a second target pixel, traverses the second target pixel, and determines whether the depth value of the traversed second target pixel is less than the depth value of the corresponding first target pixel; if the depth value of the traversed second target pixel is less than the depth value of the corresponding first target pixel, the depth value of the corresponding first target pixel is changed to the depth value of the traversed second target pixel; if the depth value of the traversed second target pixel is not less than the depth value of the corresponding first target pixel, the depth value of the corresponding first target pixel remains unchanged.

[0177] In this embodiment, by comparing the depth values ​​of the first target pixel and the second target pixel at the same pixel location, the occlusion relationship between the original point cloud and the target model is determined. The depth of the pixels corresponding to each data point of the occluded original point cloud is updated. The data processing is simple and efficient, providing conditions for efficient and accurate point cloud simulation.

[0178] In one embodiment, the point cloud generation method specifically includes:

[0179] The computer device acquires the Cartesian coordinates (x, y, z) of each data point in the original point cloud of the autonomous driving scenario. For example, the distribution of the original point cloud in the autonomous driving scenario is as follows: Figure 6 As shown.

[0180] According to the mapping relationship expressed by formulas (1) to (3), the coordinate data of each data point in the Cartesian coordinate system is mapped to the spherical coordinate data in the spherical coordinate system, and the azimuth angle of each data point in the original point cloud in the spherical coordinate system is obtained. Zenith angle (θ) and radial distance (r). Cartesian coordinates (x, y, z) and spherical coordinates are retained for each data point. The correspondence.

[0181] Obtain the spherical projection parameters: depth map width (W), depth map height (H), and maximum azimuth angle. Minimum azimuth angle Maximum zenith angle (θ) max ) and minimum zenith angle (θ) min Based on the azimuth angles of each data point in the original point cloud in the spherical coordinate system... The zenith angle (θ) and radial distance (r) are used to calculate the first image coordinate value (u) and the second image coordinate value (v) using formulas (4) and (5) respectively, thus obtaining the first image coordinate data (u, v). The spherical coordinate data for each data point is retained. The correspondence between the coordinates and the coordinates (u, v) of the first image after projection.

[0182] The pixel points projected onto the first depth map from the data points are determined according to the first image coordinate data, and the corresponding radial distance from the first image coordinate data is used as the pixel value of that pixel. Other pixels in the first depth map that are not projected are set to zero. For example, Figure 6 The first depth map obtained by projecting the original point cloud data points onto a spherical surface is shown below. Figure 7 As shown.

[0183] Obtain the vehicle model and its placement space relative to the autonomous driving scenario, and determine the Cartesian coordinates (x', y', z') of the points of the vehicle model in the Cartesian coordinate system of the original point cloud.

[0184] Using formulas (8) and (9), the point-spherical coordinate azimuth angle of the car model mapped to the spherical coordinate system is calculated. The zenith angle (θ') of the point ball coordinates.

[0185] Based on the coordinates and azimuth of the penalty kick The zenith angle (θ') of the point ball coordinates is used to calculate the coordinate values ​​of the third image (u') and the fourth image (v') using formulas (10) and (11) respectively, thus obtaining the coordinate data of the second image (u', v').

[0186] The plane image obtained by spherical projection of the points of the car model is determined according to the coordinate data of the second image. The plane is colored, the depth buffer is determined during the coloring process, and the depth value corresponding to the coordinate data of the second image is read from the depth buffer.

[0187] The pixel points projected onto the second depth map are determined based on the second image coordinate data. The depth value corresponding to the depth buffer of the second image coordinate data is used as the pixel value of that pixel. Other unprojected pixels in the second depth map are set to zero. For example, the second depth map is obtained by spherical projection of the points of a car model. Figure 8 As shown.

[0188] In the first depth map, the pixel corresponding to the same first image coordinate data (u, v) as the second image coordinate data (u', v') is the first target pixel; in the second depth map, the pixel corresponding to the same second image coordinate data (u', v') as the first image coordinate data (u, v) is the second target pixel. The first target pixels are traversed, and it is determined whether the depth value of the traversed first target pixel is greater than the depth value of the corresponding second target pixel. If the depth value of the traversed first target pixel is greater than the depth value of the corresponding second target pixel, the depth value of the traversed first target pixel is changed to the depth value of the corresponding second target pixel; if the depth value of the traversed first target pixel is not greater than the depth value of the corresponding second target pixel, the depth value of the traversed first target pixel remains unchanged.

[0189] For each target pixel whose depth value has been updated, the first image coordinate data and the updated depth value in the Cartesian coordinate system are calculated using the inverse operation of formulas (1) to (5). Simultaneously, the corresponding spherical coordinate data is found for the first target pixel using the first image coordinate data, and then the corresponding Cartesian coordinate data is found, as well as the data point corresponding to the Cartesian coordinate data is determined. The Cartesian coordinate data of the corresponding data point is replaced with the coordinate data obtained through the inverse operation, thereby updating the original point cloud and finally obtaining a new point cloud. For example, in an autonomous driving scenario, adding a car model yields a new point cloud as shown below. Figure 9 As shown, a car model is added to the autonomous driving scenario, where 901 is the part of the original point cloud that is replaced, and the rest are new point clouds.

[0190] In this embodiment, the original point cloud of the autonomous driving scenario and the points of the car model are spherically projected. The projection process uses the obtained spherical projection parameters of the original point cloud, eliminating the need for additional sensor parameters. Furthermore, the projection surface formed by the spherical projection has a wider field of view. The depth values ​​of the corresponding first target pixels are updated based on the second target pixels of the car model, and the data points obtained by the inverse projection processing of the depth-updated first target pixels replace the data points in the original point cloud, generating a new point cloud. Compared to the original point cloud, only the necessary depth value updates are performed, while retaining the original point cloud without depth value updates. The resulting new point cloud has a wider field of view, requires fewer additional parameters, and involves a relatively smaller amount of updated data. Compared to the method using a camera model, this method is more efficient in acquiring depth maps, and consequently, more efficient in point cloud simulation.

[0191] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0192] Based on the same inventive concept, this application also provides a point cloud generation apparatus for implementing the point cloud generation method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more point cloud generation apparatus embodiments provided below can be found in the limitations of the point cloud generation method described above, and will not be repeated here.

[0193] In one embodiment, such as Figure 10 As shown, a point cloud generation device 1000 is provided, including: a point cloud spherical projection module 1010, a target model spherical projection module 1020, a depth value update module 1030, and a point cloud update module 1040, wherein:

[0194] The point cloud spherical projection module 1010 is used to project the original point cloud of the target scene into a first depth map according to the target spherical projection parameters.

[0195] The target model spherical projection module 1020 is used to project the target model into a second depth map based on the placement space information of the target model relative to the target scene and according to the target spherical projection parameters; the second depth map includes the target depth region corresponding to the target model obtained by projection.

[0196] The depth value update module 1030 is used to determine the first target pixel point projected from the first depth map, such that the second target pixel point in the second depth map with the same position as the first target pixel point belongs to the target depth region; and to update the depth value of the first target pixel point according to the depth value of the corresponding second target pixel point.

[0197] The point cloud update module 1040 is used to update the original point cloud and obtain a new point cloud based on the difference point cloud obtained by inverse projection processing of the first target pixel after the depth value update.

[0198] In one embodiment, the point cloud spherical projection module 1010 includes a point cloud acquisition module, an angle data mapping module, and a first depth map generation module. The point cloud acquisition module acquires the spherical coordinate data of each data point in the original point cloud of the target scene, whereby the spherical coordinate data includes angle data and radial distance. The angle data mapping module maps the angle data of each data point to first image coordinate data according to the target spherical projection parameters. The first depth map generation module determines the corresponding radial distance of each first image coordinate data point and generates a first depth map based on the radial distance of each first image coordinate data point and the corresponding depth value.

[0199] In one embodiment, the point cloud acquisition module is further used to acquire the Cartesian coordinate data of each data point in the original point cloud of the target scene; and to perform coordinate system transformation on the Cartesian coordinate data according to the mapping relationship between the Cartesian coordinate system and the spherical coordinate system to obtain the spherical coordinate data of each data point in the original point cloud.

[0200] In one embodiment, the target spherical projection parameters include depth map side length data and angle range data. The angle data mapping module is further used to perform linear mapping on the angle data of each data point in the original point cloud according to the angle ratio of the corresponding angle data to the angle range represented by the angle range data and the depth map side length data, so as to obtain the first image coordinate data corresponding to each data point.

[0201] In one embodiment, the target model spherical projection module 1020 includes a target model point determination module, a point angle data mapping module, and a second depth map generation module. The target model point determination module is used to determine the point-to-sphere coordinate angle data of points on the target model based on the placement space information of the target model relative to the target scene. The point angle data mapping module is used to map the point-to-sphere coordinate angle data into second image coordinate data according to the target spherical projection parameters. The second depth map generation module is used to obtain the depth value corresponding to each second image coordinate data and generate a second depth map based on the second image coordinate data and the corresponding depth value.

[0202] In one embodiment, the second depth map generation module is further configured to determine the depth buffer used when coloring the projected target model; read the depth value corresponding to each second image coordinate data from the depth buffer; and generate a second depth map based on each second image coordinate data and the corresponding depth value.

[0203] In one embodiment, the depth value update module 1030 is further configured to compare the depth value of the first target pixel with the depth value of the corresponding second target pixel; and update the depth value of the first target pixel to the depth value of the corresponding second target pixel when the depth value of the first target pixel is greater than the depth value of the corresponding second target pixel.

[0204] The aforementioned point cloud generation device performs spherical projection on the original point cloud of the target scene and the points of the target model, respectively. The projection process uses the obtained spherical projection parameters of the original point cloud, eliminating the need for additional sensor parameters. Furthermore, the projection surface formed by the spherical projection has a wider field of view. The device also updates the depth values ​​of the corresponding first target pixels based on the second target pixel in the target depth region of the target model, and performs inverse projection processing on the first target pixels after the depth value update to obtain a difference point cloud. Based on the difference point cloud, the original point cloud is updated to generate a new point cloud. The generated new point cloud only undergoes the necessary depth value updates compared to the original point cloud, retaining the original point cloud without depth value updates. The resulting new point cloud has a wider field of view, requires fewer additional parameters, and involves a relatively smaller amount of updated data. Compared to methods using camera models, this method is more efficient in acquiring depth maps, and consequently, more efficient in point cloud simulation.

[0205] Each module in the aforementioned point cloud generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0206] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 11 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a point cloud generation method. The display unit of the computer device is used to form a visually visible image. It can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0207] Those skilled in the art will understand that Figure 11 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0208] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0209] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0210] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0211] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data shall comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0212] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0213] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0214] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A point cloud generation method, characterized in that, The method includes: The original point cloud of the target scene is projected into a first depth map according to the target spherical projection parameters; Based on the placement space information of the target model relative to the target scene, the target model is projected into a second depth map according to the target spherical projection parameters; the second depth map includes the target depth region corresponding to the target model obtained by projection; Determine a first target pixel point projected from the first depth map, such that a second target pixel point in the second depth map, which has the same position as the first target pixel point, belongs to the target depth region; Update the depth value of the first target pixel according to the depth value of the corresponding second target pixel; Based on the difference point cloud obtained by projecting the first target pixel after depth value update, the original point cloud is updated to obtain a new point cloud.

2. The method according to claim 1, characterized in that, The step of projecting the original point cloud of the target scene into a first depth map according to the target spherical projection parameters includes: Obtain spherical coordinate data of each data point in the original point cloud of the target scene, wherein the spherical coordinate data includes angle data and radial distance; According to the target spherical projection parameters, the angle data of each data point is mapped to the first image coordinate data; Determine the radial distance corresponding to each of the first image coordinate data, and generate a first depth map based on the radial distance of each of the first image coordinate data and the corresponding depth representation value.

3. The method according to claim 2, characterized in that, The acquisition of spherical coordinate data of each data point in the original point cloud of the target scene includes: Obtain the Cartesian coordinates of each data point in the original point cloud of the target scene; According to the mapping relationship between the Cartesian coordinate system and the spherical coordinate system, the Cartesian coordinate data is transformed to obtain the spherical coordinate data of each data point in the original point cloud.

4. The method according to claim 2 or 3, characterized in that, The target spherical projection parameters include depth map side length data and angle range data; mapping the angle data to first image coordinate data according to the target spherical projection parameters includes: For the angle data of each data point in the original point cloud, a linear mapping is performed according to the proportion of the corresponding angle data to the angle range represented by the angle range data and the side length data of the depth map to obtain the first image coordinate data corresponding to each data point.

5. The method according to claim 1, characterized in that, The step of projecting the target model into a second depth map based on the placement space information of the target model relative to the target scene and according to the target spherical projection parameters includes: Based on the placement space information of the target model relative to the target scene, determine the point ball coordinate angle data of the target model's points; According to the target spherical projection parameters, the point ball coordinate angle data is mapped to the second image coordinate data; Obtain the depth value corresponding to each second image coordinate data, and generate a second depth map based on the second image coordinate data and the corresponding depth value.

6. The method according to claim 5, characterized in that, The step of obtaining the depth value corresponding to each second image coordinate data and generating a second depth map based on the second image coordinate data and the corresponding depth value includes: Determine the depth buffer to use when shading the projected target model; Read the depth value corresponding to each second image coordinate data from the depth buffer; A second depth map is generated based on the coordinate data of each second image and its corresponding depth value.

7. The method according to any one of claims 1 to 6, characterized in that, The step of updating the depth value of the first target pixel according to the depth value of the corresponding second target pixel includes: Compare the depth value of the first target pixel with the corresponding depth value of the second target pixel; If the depth value of the first target pixel is greater than the depth value of the corresponding second target pixel, the depth value of the first target pixel is updated to the depth value of the corresponding second target pixel.

8. A point cloud generation device, characterized in that, The device includes: The point cloud spherical projection module is used to project the original point cloud of the target scene into a first depth map according to the target spherical projection parameters; The target model spherical projection module is used to project the target model into a second depth map based on the placement space information of the target model relative to the target scene and according to the target spherical projection parameters; the second depth map includes the target depth region corresponding to the target model obtained by projection; The depth value update module is used to determine the first target pixel point projected from the first depth map, such that the second target pixel point in the second depth map with the same position as the first target pixel point belongs to the target depth region; and to update the depth value of the first target pixel point according to the depth value of the corresponding second target pixel point. The point cloud update module is used to update the original point cloud based on the difference point cloud obtained by inverse projection processing of the first target pixel point after depth value update, so as to obtain a new point cloud.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images