Point cloud color rendering method, device, computer equipment and storage medium

Abstract

The present application relates to a point cloud color rendering method, device, computer equipment and storage medium. The method includes: acquiring point cloud data, constructing a voxel map based on a sparse voxel structure; extracting point cloud edge features of the point cloud, and performing ray tracing on each pixel in a pre-acquired photo according to the initial posture of the photo to obtain the voxel observed by the photo; the observed voxel is the voxel with the minimum depth; correcting the posture information of the photo according to the point cloud edge features and the two-dimensional edge features of the photo; projecting all point clouds in the voxel with the minimum depth onto the photo with the corrected posture to obtain a minimum depth map, determining the point cloud sequence number to be rendered from the minimum depth map, and rendering the point cloud corresponding to the point cloud sequence number using the color information of the pixel points on the photo. The use of this method can improve the comprehensiveness and accuracy of point cloud rendering.

Description

Technical Field

[0001] The present application relates to the technical field of point cloud rendering, and in particular to a point cloud color rendering method, device, computer equipment and storage medium. Background Art

[0002] Color point cloud is one of the important products of 3D reconstruction. It can not only show the specific position of environmental objects in three-dimensional space, but also reflect the color texture of the environment. Color point cloud has a wide range of applications in 3D map generation, surveying and mapping modeling, land resources, disaster prevention and relief, and cultural relics protection.

[0003] For the point cloud coloring problem, considering that point clouds are often obtained by stitching together several frames of dedistorted radar point cloud frames or depth maps of depth cameras through pose space transformation, some coloring schemes use a single frame of point cloud data to colorize with the most recent photo. For observations at the same spatial position, the camera only needs one photo, while the lidar or depth camera needs several frames of point cloud frames at different times to obtain a dense point cloud at that position. Therefore, for observations at the same spatial position, the time when the lidar or depth camera and the camera obtain observation data is different. For data at the same time for both, the observation of the same position by the lidar or depth camera is insufficient. The single point cloud frame-photo frame coloring method will result in many point clouds at the same spatial position not being colored. Therefore, for each frame of point cloud data, a frame of photos with the same FoV needs to be saved to achieve sufficient coloring, which means that a large number of photos and point cloud frames with redundant information need to be saved. Summary of the invention

[0004] Based on this, it is necessary to provide a point cloud color rendering method, device, computer equipment and storage medium to address the above technical problems.

[0005] A point cloud color rendering method, the method comprising:

[0006] Acquire point cloud data, and construct a voxel map according to a sparse voxel structure; the voxel map retrieves voxels by given coordinates, and the voxels include multiple point clouds;

[0007] Extracting point cloud edge features of the point cloud, and performing ray tracing on each pixel in the pre-acquired photo according to the initial pose of the photo, to obtain voxels observed by the photo; the observed voxels are voxels with minimum depth;

[0008] Correcting the pose information of the photo according to the edge features of the point cloud and the two-dimensional edge features of the photo;

[0009] Project all point clouds in the minimum depth voxel onto the photo of the corrected posture to obtain a minimum depth map, determine the point cloud number to be rendered from the minimum depth map, and render the point cloud corresponding to the point cloud number using the color information of the pixel points on the photo.

[0010] In one embodiment, the hash coordinates of the voxels in the voxel map are:

[0011] hash_id=(vidz*Q+vidy)*Q+vidx

[0012] Among them, (vidx, vidy, vidz) represents the voxel coordinates, Q is a prime number whose value exceeds the set value, and hash_id represents the hash coordinates;

[0013] The index relationship between the hash coordinates and the voxel map is:

[0014] V=M[hash_id]

[0015] The side length of a voxel is set to voxelSize, and the point cloud in the voxel satisfies the following relationship:

[0016] double vidx=x / voxelSize

[0017] double vidy=y / voxelSize

[0018] double vidz=z / voxelSize

[0019] if (vidz<0) vidz-=2

[0020] if (vidx<0) vidx-=2

[0021] if (vidy<0) vidy-=2

[0022] Among them, (x, y, z) represents the point cloud coordinates, and vid represents the voxel coordinates of the voxel to be solved.

[0023] In one embodiment, it also includes:

[0024] Iteratively select any three points in the voxel that constitute the plane, and determine the point cloud in the voxel that belongs to the current plane according to the plane equation that constitutes the plane, until the plane to which each point cloud in the voxel belongs is recorded; determine the intersection line of the intersecting planes in the voxel, and determine whether the intersection line passes through the voxel; if so, record it as an edge point according to the point cloud in the voxel that is closest to the intersection line.

[0025] In one embodiment, it also includes:

[0026] Get the boundary information of the voxel as Xmin, Xmax, Ymin, Ymax, Zmin, Zmax;

[0027] Substitute Xmin and Xmax into the plane equations of the two planes to obtain points E1 and E2 on the intersection line;

[0028] Determine the maximum and minimum Y coordinates of points E1 and E2 are ymax, ymin, if ymax>=Ymin and ymin<=Ymax, then the intersection line is within the Y axis coordinate range of the voxel;

[0029] Determine the maximum and minimum Z coordinates of points E1 and E2 are zmax, zmin, if zmax>=Zmin and zmin<=Zmax, then the intersection line is within the Z axis coordinate range of the voxel;

[0030] If the intersection line is within both the Y-axis and Z-axis coordinate ranges of the voxel, the intersection line passes through the voxel.

[0031] In one of the embodiments, it also includes: taking E1 as the starting point, view=E2E1 / |E2E1| as the direction, and point cloud density s as the step length, searching from E1 to E2 in sequence, if there is a point P in the current voxel within the range of s around the position Ek=E1+view*k*s, then P is an edge point.

[0032] In one embodiment, it also includes

[0033] The brightness of the photo is normalized as follows:

[0034]

[0035] Wherein, γ represents the gamma value, G represents the average grayscale value after the photo is converted into a grayscale image, and B represents the brightness after modification;

[0036] According to the gamma value, the brightness of the pixels in the photo is normalized as follows:

[0037]

[0038] Among them, V represents the original value of the pixel brightness, V new Indicates the updated brightness.

[0039] The photo is converted from RGB space to LAB space, and the hue of the photo is normalized as follows:

[0040]

[0041] Here, ave refers to the mean value of the whole photo calculated in the A and B channels of the LAB space under this channel;

[0042] After the update is completed, convert the LAB space to RGB space.

[0043] In one of the embodiments, the method further includes: obtaining the pose information of the photo, determining the starting point of ray tracing according to the pose information in a point cloud coordinate system, and determining the voxel where the ray first collides as the minimum depth voxel.

[0044] A point cloud color rendering device, the device comprising:

[0045] A map construction module, used for acquiring point cloud data and constructing a voxel map according to a sparse voxel structure; the voxel map retrieves voxels by given coordinates, and the voxels include multiple point clouds;

[0046] An edge feature extraction module is used to extract the point cloud edge features of the point cloud, and to perform ray tracing on each pixel in the pre-acquired photo according to the initial posture of the photo to obtain the voxel observed by the photo; the observed voxel is the voxel with the minimum depth;

[0047] A posture correction module, used to correct the posture information of the photo according to the edge features of the point cloud and the two-dimensional edge features of the photo;

[0048] A rendering module is used to project all point clouds in the minimum depth voxel onto the photo of the corrected posture to obtain a minimum depth map, determine the point cloud number to be rendered from the minimum depth map, and render the point cloud corresponding to the point cloud number using the color information of the pixel points on the photo.

[0049] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0050] Acquire point cloud data, and construct a voxel map according to a sparse voxel structure; the voxel map retrieves voxels by given coordinates, and the voxels include multiple point clouds;

[0051] Extracting point cloud edge features of the point cloud, and performing ray tracing on each pixel in the pre-acquired photo according to the initial pose of the photo, to obtain voxels observed by the photo; the observed voxels are voxels with minimum depth;

[0052] Correcting the pose information of the photo according to the edge features of the point cloud and the two-dimensional edge features of the photo;

[0053] Project all point clouds in the minimum depth voxel onto the photo of the corrected posture to obtain a minimum depth map, determine the point cloud number to be rendered from the minimum depth map, and render the point cloud corresponding to the point cloud number using the color information of the pixel points on the photo.

[0054] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0055] Acquire point cloud data, and construct a voxel map according to a sparse voxel structure; the voxel map retrieves voxels by given coordinates, and the voxels include multiple point clouds;

[0056] Extracting point cloud edge features of the point cloud, and performing ray tracing on each pixel in the pre-acquired photo according to the initial pose of the photo, to obtain voxels observed by the photo; the observed voxels are voxels with minimum depth;

[0057] Correcting the pose information of the photo according to the edge features of the point cloud and the two-dimensional edge features of the photo;

[0058] Project all point clouds in the minimum depth voxel onto the photo of the corrected posture to obtain a minimum depth map, determine the point cloud number to be rendered from the minimum depth map, and render the point cloud corresponding to the point cloud number using the color information of the pixel points on the photo.

[0059] The above point cloud color rendering method, device, computer equipment and storage medium are different from the traditional idea of ​​coloring a single point cloud. Instead, the stitched point cloud is completely rendered. Specifically, a voxel map is constructed from the complete point cloud, and then the edge features of the point cloud are extracted based on the voxel map. The minimum depth voxel to be colored is determined through the given pose information in several photos, and then the point cloud to be rendered is rendered based on the pixel color on the photo. Through this method, accurate coloring of point clouds can be achieved without photos with a lot of redundant information. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 A schematic diagram of a process of a point cloud color rendering method in one embodiment;

[0061] Figure 2 is a structural block diagram of a point cloud color rendering device in one embodiment;

[0062] Figure 3 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0064] In one embodiment, Figure 1 As shown, a point cloud color rendering method is provided, comprising the following steps:

[0065] Step 102, obtaining point cloud data, and constructing a voxel map according to the sparse voxel structure.

[0066] The voxel map retrieves a voxel by a given coordinate, and a voxel contains multiple point clouds.

[0067] Step 104 , extracting the point cloud edge features of the point cloud, and performing ray tracing on each pixel in the pre-acquired photo according to the initial pose of the photo, to obtain the voxel observed by the photo.

[0068] The observed voxel is the voxel with the smallest depth.

[0069] Step 106, correcting the pose information of the photo according to the edge features of the point cloud and the two-dimensional edge features of the photo.

[0070] In this step, the pose given by the photo is often inaccurate, which is manifested as the point cloud is not aligned with the corresponding texture of the photo when it is projected onto the current photo. The reasons for this problem may be: the photo timestamp is not aligned with the pose timestamp, the pose used is not strictly the pose that actually corresponds to the photo; the pose itself has noise / error, such as the pose provided by GPS positioning. Therefore, in this case, the photo pose needs to be corrected.

[0071] In this step, the pose of the photo can be corrected by using the two-dimensional edge features and point cloud edge features in the photo.

[0072] Step 108, project all point clouds in the minimum depth voxel onto the photo of the corrected posture to obtain a minimum depth map, determine the point cloud number to be rendered from the minimum depth map, and render the point cloud corresponding to the point cloud number using the color information of the pixel points on the photo.

[0073] The above point cloud color rendering method is different from the traditional idea of ​​coloring a single point cloud. Instead, the stitched point cloud is completely rendered. Specifically, a voxel map is constructed from the complete point cloud, and then the edge features of the point cloud are extracted based on the voxel map. The minimum depth voxel to be colored is determined through the given pose information in several photos, and then the point cloud to be rendered is rendered based on the pixel color on the photo. Through this method, accurate coloring of point clouds can be achieved without photos with a lot of redundant information.

[0074] In one embodiment, for step 102, constructing a voxel map specifically includes:

[0075] Get the point cloud data and build a voxel map based on the sparse voxel structure; the hash coordinates of the voxels in the voxel map are:

[0076] hash_id=(vidz*Q+vidy)*Q+vidx

[0077] Among them, (vidx, vidy, vidz) represents the voxel coordinates, Q is a prime number whose value exceeds the set value, and hash_id represents the hash coordinates.

[0078] The relationship between hash coordinates and voxel map indexes is:

[0079] V=M[hash_id]

[0080] Set the side length of the voxel to voxelSize, and the point cloud in the voxel satisfies the following relationship:

[0081] double vidx=x / voxelSize

[0082] double vidy=y / voxelSize

[0083] double vidz=z / voxelSize

[0084] if (vidz<0) vidz-=2

[0085] if (vidx<0) vidx-=2

[0086] if (vidy<0) vidy-=2

[0087] Among them, (x, y, z) represents the point cloud coordinates, and vid represents the voxel coordinates of the voxel to be solved.

[0088] In another embodiment, for step 104, the step of extracting point cloud edge features of the point cloud includes:

[0089] Iteratively select any three points in the voxel that constitute the plane, and determine the point cloud in the voxel that belongs to the current plane according to the plane equation that constitutes the plane, until the plane to which each point cloud in the voxel belongs is recorded; determine the intersection line of the intersecting planes in the voxel, and determine whether the intersection line passes through the voxel; if so, record it as an edge point based on the point cloud in the voxel that is closest to the intersection line.

[0090] Specifically, take any three points A, B, and C in the voxel, then these three points can uniquely determine a plane, the normal vector of the plane is: n = AB × AC, the plane equation is n(1)*(X-Xa)+n(2)*(Y-Ya)+n(3)*(Z-Za)=0, traverse the remaining points P in the voxel, if the absolute value of n(1)*(Xp-Xa)+n(2)*(Yp-Ya)+n(3)*(Zp-Za) is less than the threshold 1, then P is an inner point, count the number of inner points of the plane ABC, repeat the above process, record and update the plane ABC with the most inner points, if after multiple cycles, the plane solution with the most inner points is not updated, then the plane is a plane in the voxel, and all the corresponding inner points are points on the current plane. For the remaining point clouds, re-extract the plane until all the point clouds in the voxel belong to a certain plane.

[0091] In another embodiment, it is necessary to determine whether the intersection line between the planes in the voxel passes through the voxel, and the specific method is as follows: obtain the boundary information of the voxel as Xmin, Xmax, Ymin, Ymax, Zmin, Zmax; substitute Xmin, Xmax into the plane equations of the two planes to obtain points E1 and E2 on the intersection line; determine the maximum and minimum values ​​of the Y coordinates in points E1 and E2 as ymax, ymin, if ymax>=Ymin and ymin<=Ymax, then the intersection line is within the Y-axis coordinate range of the voxel; determine the maximum and minimum values ​​of the Z coordinates in points E1 and E2 as zmax, zmin, if zmax>=Zmin and zmin<=Zmax, then the intersection line is within the Z-axis coordinate range of the voxel; if the intersection line is within the Y-axis and Z-axis coordinate ranges of the voxel at the same time, then the intersection line passes through the voxel.

[0092] If the intersection line passes through the voxel, take E1 as the starting point, view = E2E1 / |E2E1| as the direction, point cloud density s as the step size, and search from E1 to E2 in sequence. If there is a point P in the current voxel within the range of s around the position Ek = E1 + view*k*s, then P is an edge point.

[0093] By repeating the above process, all edge points in the voxel and the search for edge points in all voxels can be recorded.

[0094] In one embodiment, the photo needs to be preprocessed, which mainly includes two processes: normalizing the brightness of the photo as follows:

[0095]

[0096] Among them, γ represents the gamma value, G represents the average grayscale value after the photo is converted into a grayscale image, and B represents the modified brightness; according to the gamma value, the brightness of the pixels in the photo is normalized as follows:

[0097]

[0098] Among them, V represents the original value of the pixel brightness, V new Indicates the updated brightness.

[0099] The photo is converted from RGB space to LAB space, and the hue of the photo is normalized as follows:

[0100]

[0101] Here, ave refers to the average value of the whole photo in the A and B channels of the LAB space; after the update is completed, the LAB space is converted to RGB space. Obviously, after the brightness is normalized, it is easier to assign the color of the pixel to the point cloud.

[0102] In one of the embodiments, the pose information of the photo is obtained, and in the point cloud coordinate system, the starting point of ray tracing is determined according to the pose information to perform a search, and the body that the ray first hits is determined as the body with the minimum depth.

[0103] Specifically, for a certain pixel coordinate u, under a given camera model π, the ray of the observation direction corresponding to the pixel in the camera coordinate system is d C =π -1 (u), the current photo pose is known Then in the point cloud coordinate system, the ray starts from Direction by Starting from d W The search is forward in the direction with voxelSize as the step size. If a voxel V is encountered in the point cloud M for the first time, V is the voxel with the minimum depth observed by the current photo. The voxel is recorded and the search stops in the direction of the pixel. This operation is performed for all pixel directions to find all the voxels with the minimum depth.

[0104] In one of the embodiments, the posture in the photo also needs to be corrected, specifically in the following manner:

[0105] For the current photo, first extract the 2D edge, one method is to extract the Canny edge. For the minimum depth voxel, extract the 3D point P marked as the edge and project it onto the current photo:

[0106]

[0107] Put the 2D edge points into the kdtree, and for each projection point of the 3D point, use the kdtree to search for the nearest 2D point, and optimize the projection error r between the two points to update the pose x:

[0108]

[0109] in,

[0110]

[0111] In the above formula, X, Y, and Z are all P C The specific coordinate value in .

[0112] The translational derivative is:

[0113]

[0114] The rotating part is:

[0115]

[0116] In one of the embodiments, all points of the voxel are projected onto a photo of the corrected posture to obtain a minimum depth map, and the correspondence between the point cloud and the pixel is obtained. Construct a minimum depth map. Project all the above voxel midpoints onto the current pixel plane. If a certain point projection position has no depth or a depth value, but the current point depth value is smaller than the existing depth value, then record the depth value at the pixel and record the 3D point sequence number. Filter the obscured point cloud. Considering that even the densest point cloud is sparsely distributed in space, for the point cloud of a nearby object, when projected onto a photo, some pixels that should have been projected onto the nearby points are not projected, showing a sparse hole state. If there are also point clouds behind the nearby object at this time, these point clouds will be projected to the hole pixel position, so when coloring, there will be an error that the distant point cloud is colored with the nearby color. To avoid this problem, the depth map needs to be filtered. Traverse each pixel A of the depth map. In the area of ​​windowSize around it, if a point B is found with a depth value much greater than the current point A, delete the point B; if a point B is found with a depth value much less than the current point A, delete the current point A. The calculation formula for the search range is:

[0117]

[0118] f is the focal length of the camera, depth is the pixel depth, and ratio is the preset threshold.

[0119] After the above search, we can determine which points are colored by which pixel of the current photo. If the current point is colored for the first time, the RGB value of the pixel corresponding to the point is directly given to the point, and the depth value of the point on the current photo is recorded. The color variance of the current 3D point is determined by the depth value of the point projected onto the colored photo. The greater the depth, the greater the variance. For a 3D point with a depth of 1m, its color covariance is:

[0120]

[0121] For a 3D point with depth depth, its color covariance is cov*depth 2 .

[0122] For a 3D point that has been colored multiple times, its current color is C, the current color variance is COV, the observed color in the most recent frame is c, and the variance is cov. At this time, the color and variance of the point need to be updated using the following formula:

[0123] cov new =(cov -1 +COV -1 ) -1

[0124] c new =(cov -1 *c+COV -1 *C) -1 *cov new

[0125] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0126] In one embodiment, Figure 2 As shown, a point cloud color rendering device is provided, comprising: a map construction module 202, an edge feature extraction module 204, a posture correction module 206 and a rendering module 208, wherein:

[0127] A map construction module 202 is used to obtain point cloud data and construct a voxel map according to a sparse voxel structure; the voxel map retrieves voxels by given coordinates, and the voxels include multiple point clouds;

[0128] The edge feature extraction module 204 is used to extract the edge features of the point cloud, and perform ray tracing on each pixel in the pre-acquired photo according to the initial posture of the photo to obtain the voxel observed by the photo; the observed voxel is the voxel with the minimum depth;

[0129] A posture correction module 206, used to correct the posture information of the photo according to the edge features of the point cloud and the two-dimensional edge features of the photo, so as to iteratively extract the accurate minimum depth voxel;

[0130] The rendering module 208 is used to project all the point clouds in the minimum depth voxel onto the photo of the corrected posture to obtain a minimum depth map, determine the point cloud number to be rendered from the minimum depth map, and render the point cloud corresponding to the point cloud number using the color information of the pixel points on the photo.

[0131] For the specific definition of the point cloud color rendering device, please refer to the definition of the point cloud color rendering method above, which will not be repeated here. Each module in the above-mentioned point cloud color rendering device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0132] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 3 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a point cloud color rendering method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0133] Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0134] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method in the above embodiment when executing the computer program.

[0135] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiment are implemented.

[0136] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0137] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0138] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A point cloud color rendering method, characterized in that: The method comprises: Acquire point cloud data, and construct a voxel map according to a sparse voxel structure; the voxel map retrieves voxels by given coordinates, and the voxels include multiple point clouds; Extracting point cloud edge features of the point cloud, and performing ray tracing on each pixel in the pre-acquired photo according to the initial pose of the photo, to obtain voxels observed by the photo; the observed voxels are voxels with minimum depth; Correcting the pose information of the photo according to the edge features of the point cloud and the two-dimensional edge features of the photo; Projecting all point clouds in the minimum depth voxel onto the photo of the corrected posture to obtain a minimum depth map, determining the point cloud number to be rendered from the minimum depth map, and rendering the point cloud corresponding to the point cloud number using the color information of the pixel points on the photo; Extracting point cloud edge features of the point cloud includes: Iteratively select any three points in the voxel that constitute a plane, and determine the point cloud in the voxel that belongs to the current plane according to the plane equation that constitutes the plane, until the plane to which each point cloud in the voxel belongs is recorded; determining an intersection line of intersecting planes within the voxel, and determining whether the intersection line passes through the voxel; If so, it is recorded as an edge point according to the point cloud closest to the intersection line in the voxel.

2. The method according to claim 1, characterized in that The hash coordinates of the voxels in the voxel map are: hash_id=(vidz*Q+vidy)*Q+vidx Among them, (vidx, vidy, vidz) represents the voxel coordinates, Q is a prime number whose value exceeds the set value, and hash_id represents the hash coordinates; The index relationship between the hash coordinates and the voxel map is: V=M[hash_id] The side length of a voxel is set to voxelSize, and the point cloud in the voxel satisfies the following relationship: double vidx=x / voxelSize double vidy=y / voxelSize double vidz=z / voxelSize if (vidz<0) vidz-=2 if (vidx<0) vidx-=2 if (vidy<0) vidy-=2 Among them, (x, y, z) represents the point cloud coordinates, and vid represents the voxel coordinates of the voxel to be solved.

3. The method according to claim 1, characterized in that The determining of the intersection line of the intersecting planes in the voxel and judging whether the intersection line passes through the voxel comprises: Get the boundary information of the voxel as Xmin, Xmax, Ymin, Ymax, Zmin, Zmax; Substitute Xmin and Xmax into the plane equations of the two planes to obtain points E1 and E2 on the intersection line; Determine the maximum and minimum Y coordinates of points E1 and E2 are ymax, ymin, if ymax>=Ymin and ymin<=Ymax, then the intersection line is within the Y axis coordinate range of the voxel; Determine the maximum and minimum Z coordinates of points E1 and E2 are zmax, zmin, if zmax>=Zmin and zmin<=Zmax, then the intersection line is within the Z axis coordinate range of the voxel; If the intersection line is within both the Y-axis and Z-axis coordinate ranges of the voxel, the intersection line passes through the voxel.

4. The method according to claim 3, characterized in that According to the point cloud closest to the intersection line in the voxel, it is recorded as an edge point, including: Take E1 as the starting point, view = E2E1 / |E2E1| as the direction, point cloud density s as the step size, and search from E1 to E2 in sequence. If there is a point P in the current voxel within the range of distance s around the position Ek = E1 + view*k*s, then P is an edge point.

5. The method according to any one of claims 1 to 4, characterized in that: Before performing ray tracing on each pixel in the pre-acquired photo to obtain a voxel observed by the photo, the method further includes: The brightness of the photo is normalized as follows: Wherein, γ represents the gamma value, G represents the average grayscale value after the photo is converted into a grayscale image, and B represents the brightness after modification; According to the gamma value, the brightness of the pixels in the photo is normalized as follows: Among them, V represents the original value of the pixel brightness, V new Indicates the updated brightness; The photo is converted from RGB space to LAB space, and the hue of the photo is normalized as follows: Here, ave refers to the mean value of the whole photo calculated in the A and B channels of the LAB space under this channel; After the update is completed, convert the LAB space to RGB space.

6. The method according to any one of claims 1 to 4, characterized in that: Ray tracing is performed on each pixel in the pre-acquired photo to obtain voxels observed by the photo, including: The pose information of the photo is obtained, and in the point cloud coordinate system, the starting point of ray tracing is determined according to the pose information to perform a search, and the voxel that the ray first hits is determined as the voxel with the minimum depth.

7. A point cloud color rendering device, characterized in that: The device comprises: A map construction module, used for acquiring point cloud data and constructing a voxel map according to a sparse voxel structure; the voxel map retrieves voxels by given coordinates, and the voxels include multiple point clouds; An edge feature extraction module is used to extract the point cloud edge features of the point cloud, and to perform ray tracing on each pixel in the pre-acquired photo according to the initial posture of the photo to obtain the voxel observed by the photo; the observed voxel is the voxel with the minimum depth; A posture correction module, used to correct the posture information of the photo according to the edge features of the point cloud and the two-dimensional edge features of the photo; A rendering module is used to project all point clouds in the minimum depth voxel onto the photo of the corrected posture to obtain a minimum depth map, determine the point cloud sequence number to be rendered from the minimum depth map, and render the point cloud corresponding to the point cloud sequence number using the color information of the pixel points on the photo; The edge feature extraction module is also used to iteratively select any three points in the voxel that constitute the plane, and determine the point cloud in the voxel that belongs to the current plane according to the plane equation that constitutes the plane, until the plane to which each point cloud in the voxel belongs is recorded; determine the intersection line of the intersecting planes in the voxel, and determine whether the intersection line passes through the voxel; if so, record it as an edge point based on the point cloud in the voxel that is closest to the intersection line.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images