High-density color point cloud generation method based on projection mapping and depth prediction

Abstract

The invention discloses a high-density color point cloud generation method based on projection mapping and depth prediction, and belongs to the technical field of three-dimensional reconstruction and computer vision, and the method comprises the following steps: 1, sensor calibration: carrying out the internal reference calibration of a camera through a Zhang's calibration method, and obtaining an internal reference matrix K, determining an external parameter matrix T of the camera and the laser radar through joint calibration; in the invention, first, high-precision data fusion: through accurate sensor calibration and coordinate transformation, high-precision mapping of point cloud data and image pixels is realized, and the precision of data fusion is significantly improved; 2, high-efficiency calculation: through optimizing a depth prediction algorithm and setting an iteration convergence criterion, the method has relatively high calculation efficiency when processing large-scale data; and 3, accurate depth prediction: by introducing a gradient regularization item, depth information can be accurately predicted when a complex scene is processed, and the precision of three-dimensional reconstruction is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional reconstruction and computer vision, and in particular relates to a method for generating high-density color point clouds based on projection mapping and depth prediction. Background Art

[0002] With the rapid development of 3D reconstruction technology, the fusion of point cloud data and image data has become crucial for constructing high-precision 3D models. Point cloud data, acquired through sensors like LiDAR, provides precise 3D spatial information but lacks color and texture detail. Image data, acquired through cameras, provides rich color and texture information but lacks depth information. Therefore, effectively fusing point cloud data with image data has become a hot research topic.

[0003] Existing point cloud and image fusion methods can be divided into two main categories: geometry-based methods and deep learning-based methods. Geometry-based methods achieve data fusion by calibrating the relative positions of the camera and lidar and projecting the point cloud onto the image plane. However, these methods often rely on precise calibration results and are prone to errors when processing complex scenes. Deep learning-based methods achieve point cloud and image fusion by training neural networks to predict depth information. However, these methods require large amounts of training data and have high computational complexity.

[0004] Based on this, the present invention designs a high-density color point cloud generation method based on projection mapping and depth prediction to solve the above problems. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems in the above-mentioned background technology and to propose a high-density color point cloud generation method based on projection mapping and depth prediction.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for generating a high-density colored point cloud based on projection mapping and depth prediction includes the following steps:

[0008] Step 1, sensor calibration:

[0009] In order to associate each three-dimensional point in the point cloud with the corresponding pixel in the image, the relative position between the sensors must be determined. This requires internal calibration of each camera and external calibration between the camera and the lidar. Since the existing calibration technology is quite mature, the present invention adopts Zhang's method [46, 47] to determine the internal parameter matrix K and the external parameter matrix T. The internal parameter matrix K describes the internal imaging parameters of the camera and is mainly used to characterize its internal geometric and optical properties. The external parameter matrix T describes the relative position between the camera and the lidar, so that the observation data from different sensors can be fused and analyzed in a unified coordinate system. This matrix usually consists of a rotation matrix and a translation vector, represented as a 4×4 homogeneous transformation matrix.

[0010] The expression of the internal parameter matrix K is:

[0011]

[0012] Among them, f x and f y is the product of focal length and pixel size, representing the scaling factor in the horizontal and vertical directions, respectively, o x and o y are the pixel coordinates of the principal point, usually located at the optical center of the image.

[0013] The expression of the external parameter matrix T is:

[0014]

[0015] in, is the rotation matrix used to describe the rotation of the lidar coordinate system relative to the camera coordinate system, is a translation vector that describes the translation of the lidar coordinate system relative to the camera coordinate system.

[0016] Step 2, three-dimensional coordinate transformation

[0017] Points in the 3D scene are projected onto the 2D image plane through three coordinate transformation steps and finally mapped onto the pixel grid.

[0018] Step 2.1, rigid body transformation from world coordinate system to camera coordinate system:

[0019] In three-dimensional space, the point x in the world coordinate system is w Mapped to point x in the camera coordinate system through rigid body transformation C The transformation process is as follows:

[0020] x c =Rx w +t#(3)

[0021] where x c=[x c ,y c ,z c ] T Indicates the coordinates of the point in the camera coordinate system

[0022] Or using homogeneous coordinates:

[0023]

[0024] Step 2.2, perspective projection from the camera coordinate system to the image plane:

[0025] In the camera coordinate system, point x c =[x c ,y c ,z c ] T . Through the pinhole camera model x c Projected onto the two-dimensional image plane, the basic principle of the pinhole model is to project a three-dimensional point through the focal length f. Correspondingly, the projection equation is:

[0026]

[0027] Among them, x i and y i is the coordinate in the image plane, and f is the focal length of the camera. This projection relationship scales the x and y coordinates of the 3D point according to its depth zc, so that it can be projected from 3D space to the 2D plane.

[0028] Step 2.3, mapping from image plane to pixel coordinate system:

[0029] After perspective projection, point x i =[x i ,y i ] T Located on the image plane, it must be converted to a discrete pixel coordinate system. This process depends on the resolution of the camera sensor and the geometric mapping from the image plane to the pixel grid. The image sensor coordinate system has an origin offset. And the focal length of the camera can be expressed in pixels as f x and f y Therefore, from the image coordinate x i To pixel coordinate x p =[u p ,v p ] T The mapping is given by the following formula:

[0030]

[0031] Step 3, depth prediction

[0032] Through a series of coordinate transformations, the algorithm of the present invention accurately maps the 3D point cloud data to the 2D image plane, thereby obtaining the pixel coordinates corresponding to the point cloud data on the image plane. To facilitate the subsequent depth prediction, the depth information of the matching pixel points is first extracted and this depth value is defined as D. This process can be described as follows:

[0033]

[0034] The matched pixel information is converted from (R, G, B) to (R, G, B, D), where (R, G, B) represents the RGB color model with intensity values between 0 and 1, and D is the depth value of the 2D image.

[0035] Step 4: Pixel-space dual-constrained depth gradient adaptive regularization algorithm (PS-DGAR)

[0036] Because the number of mapped points in the 3D point cloud to 2D image mapping process is far less than the total number of image pixels, a large number of pixels lack corresponding depth information. To enhance the texture features of the region of interest, we first initialize the depth values D of the unmatched pixels to 0 and then use a depth prediction algorithm to fill these areas. Based on the predicted depth information, corresponding 3D points are generated in the point cloud space through backprojection, effectively enhancing the texture features of the region of interest.

[0037] To solve the above depth prediction problem, the present invention develops a pixel-space dual-constrained depth gradient adaptive regularization algorithm (PS-DGAR). This method fully utilizes the depth space distance and intensity difference information between pixels to perform depth prediction and has a solid theoretical basis.

[0038] Mathematical derivation process of pixel-space dual-constraint depth gradient adaptive regularization algorithm:

[0039] The algorithm first selects a pixel P in the depth map with an initial depth value of zero. r Then check the neighborhood N(P r ), where N(P r ) is defined as a fixed-size window (5×5 in this paper) containing n adjacent pixels p1, p2, …, p with non-zero depth values. n Pixel P is processed only when the number of pixels with non-zero depth value n ≥ threshold r Depth estimation is performed, where the threshold is usually a predefined constant, which is 10 in this paper. If n is less than the threshold, skip the P r This ensures that there is enough depth information in the neighborhood for accurate estimation, thus improving the robustness of the algorithm.

[0040] If the number of valid depth pixels in the neighborhood is equal to or exceeds the threshold, the pixel P is weighted averaged using a weighted average model that considers both spatial distance and feature similarity. r Perform depth estimation. Pixel P r The estimated depth D r It is given by the following formula:

[0041]

[0042] Among them D j is a known neighboring pixel P j Depth, W j is the weight associated with each neighboring pixel. j Defined as:

[0043]

[0044] Among them, |p r -p j | 2 is the center pixel p r and adjacent pixel p j The square of the intensity difference between ||p r -p j || 2 is the center pixel p r With adjacent pixels p j The square of the Euclidean distance between them; α and β are parameters that control the influence of spatial distance and feature similarity respectively; α is a normalization constant used to balance the weights.

[0045] Step 5: Depth Estimation Model

[0046] The above D r and w j The two equations ensure that pixels closer to the center pixel in both spatial and feature space contribute more significantly to depth estimation. Such models are crucial for achieving accurate depth prediction, especially in areas with texture discontinuities or gradient abrupt changes. To further improve the accuracy of depth estimation, the algorithm introduces a gradient regularization term R(D r ), which forces the depth map to smooth the changes between adjacent pixels by suppressing large gradient changes.

[0047] The expression of the gradient regularization term is given by the following formula:

[0048]

[0049] The above term penalizes the difference between adjacent pixel depth values, making the transition in the estimated depth map smoother. Therefore, the final depth estimation model is given by the following formula:

[0050]

[0051] Here, λ is a regularization parameter that controls the strength of the smoothness constraint. Larger values of λ result in smoother depth maps, while smaller values result in sharper transitions between adjacent pixels. This term can reduce noise and artifacts in depth maps, resulting in more visually consistent estimates.

[0052] Step 6, iterative convergence

[0053] To ensure that the iterative depth estimation process converges, a convergence criterion needs to be established. For each iteration k, the algorithm calculates the average change in the depth values of all pixels. The convergence criterion is:

[0054]

[0055] Where N is the total number of pixels in the image, is the depth value of pixel i at the kth iteration. When the average depth change ΔD(k) is lower than the pre-set threshold ∈, the algorithm is considered to have converged:

[0056] ΔD (k) <∈#(13)

[0057] This equation ensures that the iteration process ends when the depth map reaches a stable state, avoiding unnecessary computation and improving efficiency. Furthermore, to prevent the algorithm from falling into an infinite loop, a maximum number of iterations, K, is set. If the algorithm still fails to converge after reaching the maximum number of iterations, the iteration process is terminated, ensuring that the algorithm completes within a finite number of iterations.

[0058] Specific (PS-DGAR) algorithm overall process:

[0059] PS-DGAR algorithm based on pixel and spatial dual constraints

[0060] Input: initial depth map D (some pixel values are unknown and set to 0),

[0061] Neighborhood size N, effective depth pixel threshold T, parameters:

[0062] -Spatial weight α

[0063] - Feature similarity weight β

[0064] -Regularization weight λ

[0065] -convergence threshold ∈

[0066] - Maximum number of iterations K

[0067] Output: estimated depth map D

[0068] Initialization step: For each pixel p in the depth map D r , do the following: If D(p r )=0, then mark the pixel as uninitialized.

[0069] Main loop: Repeat

[0070] For all uninitialized pixels p r , do the following:

[0071] 1. Get pixel p r Neighborhood N(p r )

[0072] 2. Extract neighborhood N(p r ) with known depth values

[0073] 3. If the number of valid depth pixels is less than T, skip the pixel; otherwise, continue with the following operations: For the neighborhood N(p r ) for each pixel p j , do the following:

[0074] - Calculate weights

[0075] - Calculate weighted depth estimate

[0076] -Apply regularization:

[0077] -Update pixel p r Depth value

[0078] Convergence check:

[0079] Calculating depth change

[0080] If ΔD (k) <∈, then end the loop (reach the convergence condition)

[0081] Until the convergence condition is reached or the maximum number of iterations K is reached

[0082] Output: Updated depth map D

[0083] Experimental verification shows that the present invention demonstrates high data fusion accuracy and computational efficiency on multiple indoor environment datasets.

[0084] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0085] 1. The present invention has the following advantages: First, high-precision data fusion: Through precise sensor calibration and coordinate transformation, the present invention achieves high-precision mapping of point cloud data and image pixels, significantly improving the accuracy of data fusion;

[0086] Second, efficient computing: By optimizing the depth prediction algorithm and setting the iterative convergence criterion, the present invention has high computing efficiency when processing large-scale data;

[0087] Third, accurate depth prediction: By introducing the gradient regularization term, the present invention can accurately predict depth information when processing complex scenes, thereby improving the accuracy of 3D reconstruction. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 Schematic diagram of the process of associating 3D point clouds with 2D image pixels;

[0089] Figure 2 Schematic diagram of the neighborhood search and depth prediction process. DETAILED DESCRIPTION

[0090] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0091] Please see the attached Figure 1 -Attached Figure 2 The present invention provides a technical solution: a method for generating high-density color point clouds based on projection mapping and depth prediction, comprising the following steps: Step 1, sensor calibration and coordinate system - Goal: Establish an accurate mapping relationship between the camera and LiDAR to provide a basis for data fusion.

[0092] Implementation process:

[0093] Step 1.1, camera intrinsic calibration:

[0094] The Zhang's method is used to obtain the camera intrinsic parameter matrix K. In the specific implementation, a checkerboard calibration plate (such as Figure 1 As shown), by shooting the calibration plate image at multiple angles, extracting the corner coordinates, and calculating the focal length (f x ,f y ), principal point offset (o x ,o y ) and distortion coefficients. The internal parameter matrix K is finally expressed as:

[0095]

[0096] Step 1.2, Camera-LiDAR extrinsic calibration:

[0097] The extrinsic parameter matrix T (rotation matrix R and translation vector t) is determined through joint calibration. In implementation, the LiDAR and camera are fixed to the same bracket, and the 3D point cloud of the calibration plate and the corresponding image corner points are collected. R and t are optimized by minimizing the reprojection error. The extrinsic parameter matrix T is expressed as:

[0098]

[0099] Step 2: Coordinate mapping between point cloud and image pixels: Project the LiDAR point cloud onto the image plane and establish a 3D-2D correspondence.

[0100] Implementation process:

[0101] Step 2.1, rigid transformation (world coordinate system → camera coordinate system):

[0102] According to formula (4), point x in the world coordinate system is w =[x w ,y w ,z w ] T Convert to point x in the camera coordinate system c :

[0103]

[0104] Example: A point x w =[2.0m,1.5m,3.0m] T , after transformation, we get x c =[1.8m,1.6m,2.9m] T Step 2.2 Perspective Projection (Camera Coordinate System → Image Plane):

[0105] Based on formula (5), x is converted to c Project onto the image plane:

[0106]

[0107] Example: If f = 1200 pixels, the projection coordinate is x i =744 pixels, y i =662 pixels.

[0108] Step 2.3, pixel coordinate mapping (image plane → pixel coordinate system):

[0109] According to formula (6), combined with the internal parameter matrix K, the pixel coordinates [u p,v p ] T :

[0110]

[0111] Example: Finally get u p =1384,v p =1022, corresponding to the specific pixel position in the image.

[0112] Step 3: Depth information extraction and initialization

[0113] Goal: Assign depth values to matched pixels and initialize unmatched pixels to 0.

[0114] Step 3.1, deep correlation:

[0115] According to formula (7), the point cloud depth z c Mapped to image pixels to generate RGB-D data with depth information.

[0116] Example: A pixel (u p ,v p )=(500,300) corresponds to a depth of D=2.5m, and its RGB-D values are (0.7, 0.3, 0.2, 2.5).

[0117] Step 3.2, processing of unmatched points:

[0118] Initialize the depth value of unmatched pixels to 0 and mark them as “uninitialized”. For example, if the image resolution is 1920×1080, 30% of the pixels are unmatched and their depth values are set to 0.

[0119] Step 4: PS-DGAR depth prediction algorithm

[0120] Goal: Fill the depth values of unmatched pixels to increase the density of the point cloud.

[0121] Implementation process:

[0122] Step 4.1 Neighborhood search and screening:

[0123] With uninitialized pixel p r A 5×5 neighborhood window is constructed as the center. The number of non-zero depth pixels n in the neighborhood is counted. If n ≥ 10 (threshold), depth prediction is performed; otherwise, it is skipped.

[0124] Step 4.2 Weighted Depth Estimation:

[0125] According to the formula, calculate the neighborhood pixel weight w j , and weighted average to get the initial depth D r :

[0126]

[0127] Among them, α = 0.5 (controlling spatial distance weight), β = 0.3 (controlling feature similarity weight), and a is a normalization constant.

[0128] Example: A point p in the neighborhood j Depth D j =3.0m, its weight w j =0.8, final D r =2.8m.

[0129] Step 4.3 Gradient Regularization Optimization:

[0130] According to formula (11), add the smooth constraint term R(D r ), optimize depth estimation:

[0131]

[0132] Step 4.4, iterative convergence judgment:

[0133] According to formulas (12)-(13), calculate the average depth change ΔD (k) If ΔD (k) <∈=0.01m or the maximum number of iterations K=100 is reached, the algorithm is terminated.

[0134] Step 5: Back-projection to generate a high-density point cloud target: Convert the RGB-D image into a three-dimensional point cloud to improve density and texture details.

[0135] Step 5.1, pixels to camera coordinates:

[0136] According to the inverse transformation of formula (7), the pixel coordinates [u p ,v p ] T Convert the depth D to the point x in the camera coordinate system c :

[0137]

[0138] Step 5.2, camera to world coordinates:

[0139] Through the inverse transformation of the external parameter matrix T, x c Convert to point x in the world coordinate system w :

[0140] x w =R -1 (x c -t)

[0141] Effect of the embodiment:

[0142] Taking an indoor scene as an example, the original LiDAR point cloud contains 100,000 points. After processing with the PS-DGAR algorithm, the number of points increases to 500,000, and wall texture and furniture details are significantly enhanced. Depth prediction error is less than 5%, and reconstruction efficiency is improved by 40%. In terms of industrial applicability, this invention can be applied to fields such as robotic navigation, virtual reality, and architectural surveying and mapping, providing an efficient solution for high-precision 3D modeling.

[0143] In summary, the present invention achieves high-precision and high-efficiency three-dimensional color point cloud acquisition through precise sensor calibration, optimized coordinate transformation and depth prediction algorithm, and has broad application prospects.

[0144] A high-density color point cloud generation method based on projection mapping and depth prediction is suitable for the following fields:

[0145] Robot navigation: used for real-time positioning and map construction of robots in unknown environments;

[0146] Virtual Reality: Environment modeling for virtual reality applications;

[0147] Augmented Reality: used for virtual object positioning in augmented reality applications;

[0148] Autonomous driving: Environmental perception and path planning for autonomous vehicles.

[0149] The application of the method in robot navigation includes the following steps:

[0150] Obtain environmental information through cameras and lidar;

[0151] Establish the mapping relationship between point cloud and image through sensor calibration and coordinate transformation;

[0152] Depth prediction of unmatched pixels is performed using the PS-DGAR algorithm;

[0153] Generate a high-density colored point cloud through back-projection to construct an environment map.

[0154] The application of the method in virtual reality includes the following steps:

[0155] Obtain scene information through cameras and lidar;

[0156] Establish the mapping relationship between point cloud and image through sensor calibration and coordinate transformation;

[0157] Depth prediction of unmatched pixels is performed using the PS-DGAR algorithm;

[0158] Generate high-density colored point clouds through back-projection to construct virtual scenes.

[0159] The application of the method in augmented reality includes the following steps:

[0160] Obtain environmental information through cameras and lidar;

[0161] Establish the mapping relationship between point cloud and image through sensor calibration and coordinate transformation;

[0162] Depth prediction of unmatched pixels is performed using the PS-DGAR algorithm;

[0163] Generate high-density colored point clouds through back-projection to locate virtual objects.

[0164] The application of the method in autonomous driving includes the following steps:

[0165] Obtain road and obstacle information through cameras and lidar;

[0166] Establish the mapping relationship between point cloud and image through sensor calibration and coordinate transformation;

[0167] Depth prediction of unmatched pixels is performed using the PS-DGAR algorithm;

[0168] Generate high-density colored point clouds through back-projection to construct road maps.

[0169] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A method for generating high-density color point clouds based on projection mapping and depth prediction, characterized in that: The following steps are involved: Step 1, sensor calibration: perform intrinsic calibration on the camera using Zhang’s calibration method to obtain the intrinsic parameter matrix K, and determine the extrinsic parameter matrix T of the camera and lidar through joint calibration, where T is composed of the rotation matrix R and the translation vector t. The specific formula is: Among them, f x and f y is the focal length, c x and c y The main point coordinates. Step 2, cubic coordinate transformation: Step 2.1, rigid body transformation from world coordinate system to camera coordinate system: transform point P in the world coordinate system w Mapped to point P in the camera coordinate system through rigid body transformation c , the transformation formula is: P c =R·P w +t Step 2.2, perspective projection from the camera coordinate system to the image plane: point P is projected onto the image plane through the pinhole camera model. c Projected onto the two-dimensional image plane, the projection formula is: Among them, (u,v) is the image plane coordinate, (x c ,y c ,z c ) are the point coordinates in the camera coordinate system. Step 2.3, mapping from image plane to pixel coordinate system: Convert image plane coordinates (u, v) to pixel coordinates (u', v'). The specific formula is: Among them, s x and s y is the pixel size. Step 3, depth prediction: Depth prediction of unmatched pixels is performed using the pixel-space dual-constrained depth gradient adaptive regularization algorithm (PS-DGAR). The specific formula is: Among them, d i is the depth value of pixel i, w j is the weight of neighborhood pixel j, and N(i) is the neighborhood window of pixel i. Step 4: Back-projection to generate high-density point cloud: transform pixel coordinates (u', v') and depth value d i Back-projection into three-dimensional space generates a high-density colored point cloud. The specific formula is: Among them, z c =d i .

2. The method for generating high-density color point clouds based on projection mapping and depth prediction according to claim 1, characterized in that: In step 1, the internal parameter matrix K is obtained by Zhang's calibration method, and the specific steps are as follows: Use a checkerboard calibration plate to capture multi-angle images; Extract the coordinates of corner points in the image; Calculate the intrinsic parameter matrix K by minimizing the reprojection error; In step 2.1, the rotation matrix R and translation vector t of the rigid body transformation are obtained by minimizing the reprojection error. The specific formula is: Among them, P c,i is a point in the camera coordinate system, P w,i is a point in the world coordinate system.

3. The method for generating high-density color point clouds based on projection mapping and depth prediction according to claim 1, characterized in that: In step 3, the weight w of the pixel-space dual-constraint depth gradient adaptive regularization algorithm (PS-DGAR) is j Calculated by the following formula: Among them, I i and I j is the intensity value of pixel i and j, P i and P j is the spatial position of pixels i and j, σ I and σ P is the control parameter.

4. The method for generating high-density color point clouds based on projection mapping and depth prediction according to claim 1, characterized in that: In step 4, the high-density point cloud generated by reverse projection is achieved by the following steps: The pixel coordinates (u', v') and the depth value d i Convert to point P in the camera coordinate system c ; Through the inverse transformation of the external parameter matrix T, P c Convert to point P in the world coordinate system w .

5. A high-density color point cloud generation system based on projection mapping and depth prediction, characterized in that: include: Sensor calibration module: used to calibrate the intrinsic and extrinsic parameters of cameras and lidars; Coordinate transformation module: used to map point cloud data from the world coordinate system to the pixel coordinate system; Depth prediction module: used to perform depth prediction on unmatched pixels using the PS-DGAR algorithm; Back projection module: used to back-project pixel coordinates and depth values into three-dimensional space to generate high-density colored point clouds.

6. The high-density color point cloud generation system based on projection mapping and depth prediction according to claim 5, characterized in that: The sensor calibration module uses Zhang's calibration method to obtain the intrinsic parameter matrix K and the extrinsic parameter matrix T.

7. The high-density color point cloud generation system based on projection mapping and depth prediction according to claim 5, characterized in that: The coordinate transformation module maps the point cloud data to the pixel coordinate system through three coordinate transformations, specifically including: Rigid body transformation: maps points in the world coordinate system to the camera coordinate system; Perspective projection: Projecting points in the camera coordinate system onto the image plane; Pixel Coordinate Mapping: Convert image plane coordinates to pixel coordinates.

8. The high-density color point cloud generation system based on projection mapping and depth prediction according to claim 5, characterized in that: The depth prediction module uses the PS-DGAR algorithm to predict the depth of unmatched pixels. The specific formula is: Among them, d i is the depth value of pixel i, w j is the weight of neighborhood pixel j, and N(i) is the neighborhood window of pixel i.

9. The high-density color point cloud generation system based on projection mapping and depth prediction according to claim 5, characterized in that: The back-projection module generates a high-density colored point cloud by the following steps: The pixel coordinates (u', v') and the depth value d i Convert to point P in the camera coordinate system c ; Through the inverse transformation of the external parameter matrix T, P c Convert to point P in the world coordinate system w .

10. A device for generating high-density color point clouds based on projection mapping and depth prediction, comprising: Camera: used to obtain image data; LiDAR: used to obtain point cloud data; Processor: configured to execute the method described in claims 1-3; Memory: used to store image data, point cloud data and generated high-density color point cloud; The system is characterized in that the camera is a four-directional camera for acquiring RGB images in four directions; the laser radar is a two-dimensional laser radar that realizes three-dimensional scanning through a turntable; and the processor generates a high-density color point cloud through the following steps: Calibrate the camera and lidar; Map point cloud data to pixel coordinate system; Depth prediction of unmatched pixels is performed using the PS-DGAR algorithm; The pixel coordinates and depth values are back-projected into three-dimensional space to generate a high-density colored point cloud.

Images