A point cloud color upsampling method and system

Abstract

The application discloses a point cloud color up-sampling method and system, which is applied to the technical field of point cloud processing and comprises the following steps: acquiring geometry information, up-sampling sparse color information, generating a coarse color set through color up-sampling, extracting high-dimensional color features based on the geometry information and a deep neural network, positioning the features of neighborhood points by taking local distance difference as a weight to obtain geometry information weighted local important color features, acquiring high-dimensional local color features based on the fusion of the high-dimensional color features and the local important color features, and refining the coarse color set through the high-dimensional local color features to obtain a fine-grained color set.

Description

Technical Field

[0001] This invention relates to the field of point cloud processing technology, and in particular to a point cloud color upsampling method and system. Background Technology

[0002] In recent years, point cloud has become a widely used original 3D data representation method, which can be directly obtained through 3D sensing technology. The application areas of point cloud include, but are not limited to: 3D immersive telepresence, 3D city reconstruction, cultural heritage reconstruction, geophysical information system, autonomous driving, simultaneous localization and mapping, virtual / augmented reality [1]. Although 3D sensing technology has made significant progress in scanning quality and acquisition cost, obtaining dense point clouds with detailed texture details is still expensive and time-consuming. Therefore, it has great application value to generate dense large-scale point clouds with detailed textures from sparse point clouds obtained by scanning through upsampling technology. Point cloud upsampling is a widely used point cloud processing technology that allows the resolution of point clouds to be adjusted without modifying hardware devices. Its main goal is to generate dense point clouds from given sparse point clouds. Existing point cloud upsampling techniques [2][3] are usually for small point clouds that only contain geometric information and ignore color information upsampling. However, large-scale point clouds lacking color information have a very poor visual experience for 3D immersive systems (VR / AR). Therefore, large-scale point cloud color upsampling has great potential significance in practical applications.

[0003] Currently, there are few studies on color upsampling of point clouds. In the experiment of MFU[4], real point clouds were upsampled, and it was mentioned that geometric upsampling was performed using a model. The color of the newly upsampled point was kept the same as the nearest point in the sparse input, resulting in a colored upsampled dense point cloud. Heimann et al.[5] proposed a frequency selective grid-to-grid resampling (FSMMR) method for color upsampling of point clouds. In this method, three-dimensional points are first projected onto a two-dimensional plane. On the two-dimensional plane, color interpolation is achieved by selecting and optimizing the frequency basis function coefficients. Finally, the color-interpolated two-dimensional points are converted into three-dimensional points by three-dimensional reconstruction. Color distortion is introduced during the conversion from two-dimensional to three-dimensional, which should lead to delay and color quality degradation when applied to large-scale colored point clouds. Dinesh et al.[6] proposed a color point cloud super-resolution algorithm, which is achieved by adding internal points and minimizing the surface normals and RGB values ​​of connected points respectively. Unlike deep learning-based methods, these optimization-based methods cannot enhance the upsampled color set by extracting meaningful high-dimensional color features through neural networks. Arshad et al. [7] proposed a novel conditional generative adversarial network, PCGAN, which creates dense point clouds with color for various objects in an unsupervised manner. The network is a tree structure consisting of a leaf output layer and a set of initial branches, which generate point clouds with increasing resolution from point vectors in each training iteration. After a fixed number of iterations, a high-resolution point cloud is generated by copying the last branch. Kim et al. [8] proposed a method of first voxelizing the point cloud data and then inputting the voxelized data into the GAN network for training. However, after voxelization, the details are blurred and the relationship between the geometry and color of the point cloud is not fully utilized, so the effect is not ideal. Naik et al. [9] used deep neural networks to upsample the geometric and color information of the point cloud separately, thereby severing the connection between the geometric and color information of the point cloud. Wang et al.

[10] proposed a real-time point cloud color upsampling model based on deep learning, which is implemented using sparse convolution and neural hidden functions. In order to maintain low complexity for real-time applications, the model will be distorted when performing color upsampling on large upsampling factors. Overall, these color upsampling methods do not adequately consider the neighborhood correlation of color information in point clouds.

[0004] [1]Liu W,Sun J,Li W,et al.Deep learning on point clouds and its application:A survey[J].Sensors,2019,19(19):4188.

[0005] [2]Yu L,Li X,Fu C W,et al.Ec-net:an edge-aware point setconsolidation network[C] / / Proceedings of the European conference on computervision(ECCV).2018:386-402.

[0006] [3]Yifan W,Wu S,Huang H,et al.Patch-based progressive 3d point setupsampling[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision andPattern Recognition.2019:5958-5967.

[0007] [4]Qian Y,Hou J,Kwong S,et al.Deep magnification-flexible upsamplingover 3d point clouds[J].IEEE Transactions on Image Processing,2021,30:8354-8367.

[0008] [5]Heimann V,Spruck A,Kaup A.Frequency-selective mesh-to-meshresampling for color upsampling of point clouds[C] / / 2021IEEE 23rdInternational Workshop on Multimedia Signal Processing(MMSP).IEEE,2021:1-6.

[0009] [6]Dinesh C,Cheung G, I V.Super-resolution of 3D color pointclouds via fast graph total variation[C] / / ICASSP 2020-2020IEEE InternationalConference on Acoustics,Speech and Signal Processing(ICASSP).IEEE,2020:1983-1987.

[0010] [7]Arshad M S,Beksi W J.A progressive conditional generativeadversarial network for generating dense and colored 3D point clouds[C] / / 2020International Conference on 3D Vision(3DV).IEEE,2020:712-722.

[0011] [8]Kim B,Han S,Yi E,et al.3D Point Cloud Upsampling and ColorizationUsing GAN[C] / / International Conference on Multi-disciplinary Trends inArtificial Intelligence.Springer,Cham,2021:1-13.

[0012] [9]S.Naik,U.Mudenagudi,R.Tabib,and A.Jamadandi,“Featurenet:Upsamplingof point cloud and it’s associated features,”in SIGGRAPH Asia 2020Posters,2020,pp.1-2.

[0013]

[10] L.Wang,M.Hajiesmaili,J.Chakareski,and R.K.Sitaraman,“Cunet:Efficient point cloud color upsampling network,”arXiv preprint arXiv:2209.06112,2022.

[0014] To overcome these shortcomings, this application proposes a point cloud color upsampling method and system, which aims to generate colored point clouds with texture detail information. Summary of the Invention

[0015] The purpose of this application is to provide a point cloud color upsampling method and system, which aims to solve the problem that most existing point cloud color information upsampling methods are based on manual feature optimization and do not fully consider the neighborhood correlation of color information in point clouds.

[0016] To achieve the above objectives, this application provides the following technical solution:

[0017] This application provides a point cloud color upsampling method, including:

[0018] Geometric information is used to upsample sparse color information, and a coarse color set is generated through color upsampling.

[0019] Based on the geometric information and deep neural network, high-dimensional color features are extracted;

[0020] By using local distance difference as a weight to locate the features of neighboring points, the locally important color features weighted by the geometric information are obtained;

[0021] Based on the fusion of the high-dimensional color features and the local important color features, high-dimensional local color features are obtained;

[0022] The coarse color set is refined by using the high-dimensional local color features to obtain a fine-grained color set.

[0023] Furthermore, the step of upsampling sparse color information using geometric information and generating a coarse color set through color upsampling specifically includes the following steps:

[0024] The color upsampling methods include: geometric distance weighted color interpolation (GDWCI) and deep learning-based color interpolation (DLCI);

[0025] The geometric distance-weighted color interpolation specifically involves: using K-NN to calculate the K nearest neighbors to the color of the point cloud to be upsampled; using the geometric distance of the K nearest neighbors as weights; and using the color information of the K nearest neighbors as interpolation information to perform color interpolation, generating a coarse color set. The calculation formula is:

[0026]

[0027] Where W∈R RN×K×1 As the weight, p a ∈R RN×K×3 The color information of the K nearest neighbor.

[0028] Furthermore, the deep learning-based color interpolation specifically involves:

[0029] The first feature is extracted from the local neighborhood obtained by K-NN using a deep neural network, and the first feature is weighted based on the geometric distance to obtain an upsampled coarse color set composed of MLPs.

[0030] Furthermore, the step of extracting high-dimensional color features based on the geometric information and deep neural network specifically includes the following steps:

[0031] Calculate the geometric information Find the K nearest neighbors of each point and generate a K nearest neighbor index;

[0032] The geometric information is processed using the K-nearest neighbor index. The points in the coarse color set The points in the data and the color features generated by convolution are grouped to obtain local color information F1 and local color features F2;

[0033] The coarse color set After copying K times and subtracting from the local color information F1, the result is concatenated with the local color information F1 to obtain the second feature F3.

[0034] The local color feature F2 and the second feature F3 are concatenated and processed by MLP to obtain the third feature F. L ∈R RN×K×C ;

[0035] Regarding the third feature F L The high-dimensional color features are obtained by performing MLP and max pooling operations.

[0036] Furthermore, the step of using local distance difference as a weight to locate the features of neighboring points and obtaining the locally important color features weighted by the geometric information specifically includes the following steps:

[0037] The geometric information Copy K times and combine with the grouped geometric information Perform a subtraction operation, process the result using an MLP, and generate distance weights.

[0038] Based on distance weight and third color feature F L Multiply to generate the local important color features.

[0039] Furthermore, the step of refining the coarse color set using the high-dimensional local color features to obtain a fine-grained color set specifically includes the following steps:

[0040] Based on the high-dimensional local color feature F E Color regression is used, that is, color offset is generated through MLP;

[0041] The coarse color set Add the color offset to obtain the fine-grained color set.

[0042] This application provides a point cloud color upsampling system, including:

[0043] Color information upsampling module: acquires geometric information to upsample sparse color information, and generates a coarse color set through color upsampling;

[0044] Color information enhancement regression module: Based on the geometric information and deep neural network, extract high-dimensional color features; use local distance difference as weight to locate the features of neighboring points, and obtain the locally important color features weighted by the geometric information; obtain high-dimensional local color features based on the fusion of the high-dimensional color features and the locally important color features; refine the coarse color set through the high-dimensional local color features to obtain a fine-grained color set.

[0045] This application provides an apparatus comprising a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a point cloud color upsampling method; the processor is configured to execute the program instructions stored in the memory to implement a point cloud color upsampling.

[0046] This application provides a storage medium storing processor-executable program instructions for executing a point cloud color upsampling method.

[0047] This application provides a point cloud color upsampling method and system, which has the following beneficial effects:

[0048] (1) This application provides more accurate and visually appealing point cloud color upsampling results by modeling the neighborhood correlation of point cloud color information, which further improves the practicality of the technology.

[0049] (2) This application introduces a deep learning-based method into point cloud color upsampling. By modifying the deep neural network through point cloud characteristics and with the help of geometric information, it effectively realizes the upsampling of point cloud color and provides high-quality color information. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating a point cloud color upsampling method according to Embodiment 1 of this application;

[0051] Figure 2This is a schematic diagram of the framework of a point cloud color upsampling method according to Embodiment 1 of this application;

[0052] Figure 3 This is a schematic diagram of the color upsampling process in Embodiment 1 of this application;

[0053] Figure 4 This is a schematic diagram of the framework of the geometric distance weighted color interpolation method in Embodiment 1 of this application;

[0054] Figure 5 This is a schematic diagram of the framework of the deep learning-based color interpolation method in Embodiment 1 of this application;

[0055] Figure 6 This is a schematic diagram of the structure of a point cloud color upsampling system according to Embodiment 2 of this application;

[0056] Figure 7 This is a schematic diagram of the color information enhancement regression module in Embodiment 2 of this application;

[0057] Figure 8 This is a schematic diagram of the device structure in Embodiment 3 of this application;

[0058] Figure 9 This is a schematic diagram of the storage medium structure of Embodiment 4 of this application. Detailed Implementation

[0059] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0060] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0061] Example 1

[0062] Please see Figure 1 This is a flowchart illustrating a point cloud color upsampling method according to Embodiment 1 of this application; the steps include:

[0063] S1: Obtain geometric information to upsample sparse color information, and generate a coarse color set through color upsampling.

[0064] In this embodiment, due to the limited capacity of the GPU, a "divide and conquer" strategy is adopted, dividing the large-scale point cloud into several patches and using a patch-based training strategy. The acquired geometric information is used as auxiliary information, and sparse color information is upsampled to generate a coarse color set.

[0065] Please see Figure 2 This is a schematic diagram of the framework of a point cloud color upsampling method according to Embodiment 1 of this application.

[0066] From sparse patches Generate dense patches Provides important guidance for color upsampling. With dense geometric information As supplementary information, sparse color information Upsampling is performed to generate a coarse color set through color upsampling. Enhance the coarse color set by extracting high-dimensional local color features. The color offset is obtained by regressing color features using color regression. (Coarse color set) Adding the color offset to obtain the fine-grained color set

[0067] Please see Figure 3 This is a schematic diagram of the color upsampling process in Embodiment 1 of this application.

[0068] To leverage geometric information for color upsampling and overcome the shortcomings of introducing deep learning methods based on optimization, a color upsampling network is designed with reference to SRCNN. Unlike SRCNN, this application proposes two color upsampling methods: Geometric Distance Weighted Color Interpolation (GDWCI) and Deep Learning-Based Color Interpolation (DLCI). The color of the point to be upsampled generates a K-nearest neighbor (KNN) network, and color interpolation is performed using the color information of existing points in the KNN network. For color interpolation, K can be any value; when K is 1, a nearest neighbor assignment algorithm is used, and when K is greater than 1, a linear interpolation algorithm is used. In this embodiment, K is set to 2, and linear interpolation is employed. For the color information of existing points, GDWCI uses three-channel information (R, G, B), while DLCI uses features extracted through deep learning.

[0069] Please see Figure 4 This is a schematic diagram of the framework of the geometric distance weighted color interpolation method in Embodiment 1 of this application.

[0070] Based on the neighborhood correlation of color information in point clouds, points that are closer in distance are more similar in color. K-NN is used to calculate the K nearest neighbors of the color in the point cloud to be upsampled. The geometric distance of these K neighbors is used as weights, and their color information is used as interpolation information to perform color interpolation, generating a coarse color set. The calculation formula is:

[0071]

[0072] Where W∈RRN×K×1 As the weight, p a ∈R RN×K×3 The color information of the K nearest neighbor.

[0073] Please see Figure 5 This is a schematic diagram of the framework of the deep learning-based color interpolation method in Embodiment 1 of this application.

[0074] Traditional geometric distance weighted interpolation methods cannot extract meaningful color features to help generate high-quality color information. Therefore, this embodiment uses a deep neural network to extract the first feature from the local neighborhood obtained by K-NN, and weights the first feature based on geometric distance to obtain an upsampled coarse color set composed of MLPs.

[0075] To obtain diverse high-dimensional color features, the dense residual blocks in the deep residual network were modified and applied to the point cloud. K-NN was added to the dense residual blocks to aggregate the local neighborhood of the point cloud, and the color information of the local neighborhood was extracted using the dense residual blocks to generate the first feature.

[0076] S2: Based on the geometric information and deep neural network, extract high-dimensional color features.

[0077] In this embodiment, the geometric information is first calculated. For each point, calculate its K nearest neighbors and generate a K nearest neighbor index. Simultaneously, use the K nearest neighbor index to analyze the geometric information. The points in the coarse color set The points in the dataset and the color features generated by convolution are grouped. Due to the coarse color set... In geometric information Generated under the guidance of [the relevant authority], the K-nearest neighbor index can be directly used for color information and color feature grouping, thereby directly obtaining local color information F1 and local color feature F2.

[0078] The coarse color set After copying K times and subtracting from the local color information F1, the result is concatenated with the local color information F1 to obtain the second feature F3. The local color feature F2 and the second feature F3 are then concatenated and processed using MLP to obtain the third feature F. L ∈R RN×K×C Finally, regarding the third feature F... L The high-dimensional color features are obtained by performing MLP and max pooling operations.

[0079] S3: Using local distance difference as a weight to locate the features of neighboring points, the geometrically weighted local important color features are obtained.

[0080] In this embodiment, the geometric information Copy K times and combine with the grouped geometric information Perform a subtraction operation, process the result using an MLP, and generate distance weights.

[0081] Based on distance weight and third color feature F L Multiply to generate the local important color features.

[0082] S4 obtains high-dimensional local color features based on the fusion of the high-dimensional color features and the local important color features.

[0083] S5: The coarse color set is refined using the high-dimensional local color features to obtain a fine-grained color set.

[0084] In this embodiment, residual learning is used to obtain the color offset instead of absolute color values ​​because absolute color values ​​have a more diverse distribution and a wider range compared to relative offsets. Therefore, based on the high-dimensional local color feature F E Color regression is used, that is, color offsets are generated through MLP.

[0085] The coarse color set Add the color offset to obtain the fine-grained color set.

[0086] In summary, Embodiment 1 of this application upsamples sparse color information based on geometric information as auxiliary information, and generates a coarse color set through color upsampling; the coarse color set is enhanced by extracting high-dimensional local color features; the color offset is obtained by regressing the color features through color regression, and the coarse color set and the color offset are added together to obtain a fine-grained color set, resulting in a more accurate result with good visual effects.

[0087] Example 2

[0088] Please see Figure 6 This is a schematic diagram of the structure of a point cloud color upsampling system according to Embodiment 2 of this application; the specific content includes:

[0089] Color information upsampling module: acquires geometric information to upsample sparse color information, and generates a coarse color set through color upsampling;

[0090] Color information enhancement regression module: Based on the geometric information and deep neural network, extract high-dimensional color features; use local distance difference as weight to locate the features of neighboring points, and obtain the locally important color features weighted by the geometric information; obtain high-dimensional local color features based on the fusion of the high-dimensional color features and the locally important color features; refine the coarse color set through the high-dimensional local color features to obtain a fine-grained color set.

[0091] In this embodiment, the system consists of a color information upsampling module and a color information enhancement and regression module. The acquired geometric information serves as auxiliary information, upsampling the sparse color information to generate a coarse color set. The color information enhancement and regression module, aided by geometric information, refines the coarse color set to obtain a fine-grained color set.

[0092] Please see Figure 7 This is a schematic diagram of the color information enhancement regression module in Embodiment 2 of this application.

[0093] The color information enhancement regression module is designed based on an improvement upon the local refinement unit in the Dis-PU, which extracts features beneficial to point cloud geometry but does not consider point cloud color information. Specifically, this application considers the neighborhood correlation of color information to enhance the coarse color set. A deep neural network is used to extract high-dimensional color features, which are then fused with locally important color features weighted by geometric information to obtain high-dimensional local color features F. E ∈R RN×C It effectively upsamples the point cloud colors, providing high-quality color information.

[0094] Furthermore, this application conducted tests on a large-scale colored point cloud upsampling dataset. For quantitative evaluation, the Peak Signal-to-Noise Ratio (PSNR) calculated using MPEG software was used as the evaluation metric to assess the upsampled color information. In the color evaluation metrics, a higher value indicates a better color upsampling result. Specifically, the PSNR values ​​of the nearest neighbor assignment method, FSMMR method, FGTV method mentioned in MFU, and the proposed LSC-PU (LSC-PU-GDWCI) method based on GDWCI and the LSC-PU (LSC-PU-DLCI) method based on DLCI were compared under different upsampling factors. The performance comparison of the color upsampling methods is shown in the table below:

[0095]

[0096] Bold values ​​indicate the best results. When the upsampling factor is 4, the point cloud PSNRs generated by the MFU, FSMMR, and FGTV methods are 31.58 dB, 28.84 dB, and 29.60 dB, respectively. The LSC-PU-GDWCI and LSC-PU-DLCI methods proposed in this application achieve 33.57 dB and 33.90 dB, respectively, under the same conditions, which are 2.32 dB, 5.06 dB, and 4.30 dB higher than MFU, FSMMR, and FGTV, respectively. When the upsampling factors are 8, 12, and 16, the optimal PSNRs achieved in this application are 32.10 dB, 31.10 dB, and 30.39 dB, respectively, significantly outperforming other methods. Furthermore, when the upsampling factors are 4, 8, 12, and 16, the LSC-PU-DLCI method outperforms the traditional interpolation method LSC-PU-GDWCI due to the extraction of meaningful high-dimensional color features by the deep neural network. Experimental results clearly demonstrate that the proposed technical solution significantly improves the PSNR value of point cloud color information compared to other methods.

[0097] In summary, Embodiment 2 of this application generates a colored point cloud with texture detail information through a color information upsampling module and a color enhancement regression module. Specifically, by modeling the neighborhood correlation of color information, color upsampling is significantly enhanced. In the point cloud color upsampling method, two color interpolation methods are proposed to generate a coarse upsampled color set, and then the color enhancement regression module is used to refine the coarse upsampled color set to generate a colored point cloud containing rich texture detail information.

[0098] Example 3

[0099] Please see Figure 8 This is a schematic diagram of the device structure in Embodiment 3 of this application. The device 50 includes a processor 51 and a memory 52 coupled to the processor 51.

[0100] The memory 52 stores program instructions for implementing the point cloud color upsampling method described above.

[0101] The processor 51 is used to execute program instructions stored in the memory 52 to implement a point cloud color upsampling.

[0102] The processor 51 can also be referred to as a CPU (Central Processing Unit).

[0103] Processor 51 may be an integrated circuit chip with signal processing capabilities. Processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. A general-purpose processor may be a microprocessor or any conventional processor.

[0104] Example 4

[0105] Please see Figure 9 This is a schematic diagram of the storage medium in Embodiment 4 of this application. The storage medium in this embodiment stores a program file 61 capable of implementing all the above methods. This program file 61 can be stored in the storage medium in the form of a software product, including several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks, or devices such as computers, servers, mobile phones, and tablets.

[0106] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, article, or method. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, article, or method that includes that element.

[0107] The above description is only a preferred embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural changes made based on the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

[0108] Although embodiments of this application have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the appended claims and their equivalents.

[0109] Of course, the present invention may have many other embodiments. Based on this embodiment, other embodiments obtained by those skilled in the art without any creative effort are all within the scope of protection of the present invention.

Claims

1. A point cloud color upsampling method, characterized in that, include: Geometric information is used to upsample sparse color information, and a coarse color set is generated through color upsampling. Based on the geometric information and deep neural network, high-dimensional color features are extracted; By using local distance difference as a weight to locate the features of neighboring points, the locally important color features weighted by the geometric information are obtained; Based on the fusion of the high-dimensional color features and the local important color features, high-dimensional local color features are obtained; The coarse color set is refined using the high-dimensional local color features to obtain a fine-grained color set; The step of extracting high-dimensional color features based on the geometric information and deep neural network specifically includes the following steps: Calculate the geometric information Find the K nearest neighbors of each point and generate a K nearest neighbor index; The geometric information is processed using the K-nearest neighbor index. The points in the coarse color set The points in the image and the color features generated by convolution are grouped to obtain local color information. and local color features ; The coarse color set Copy K times with the local color information After performing the subtraction operation, the local color information is compared with... By splicing, the second feature is obtained. ; The local color features and the second feature The third feature is obtained by splicing and performing MLP processing. ; Regarding the third feature The high-dimensional color features are obtained by performing MLP and max pooling operations.

2. The point cloud color upsampling method according to claim 1, characterized in that, The steps of upsampling sparse color information by acquiring geometric information and generating a coarse color set through color upsampling specifically include the following steps: The color upsampling methods include: geometric distance weighted color interpolation (GDWCI) and deep learning-based color interpolation (DLCI); The geometric distance-weighted color interpolation specifically involves: using K-NN to calculate the K nearest neighbors to the color of the point cloud to be upsampled; using the geometric distance of the K nearest neighbors as weights; and using the color information of the K nearest neighbors as interpolation information to perform color interpolation, generating a coarse color set. The calculation formula is: in As weight, for Color information of nearest neighbor points.

3. The point cloud color upsampling method according to claim 2, characterized in that, The deep learning-based color interpolation specifically involves: The first feature is extracted from the local neighborhood obtained by K-NN using a deep neural network, and the first feature is weighted based on the geometric distance to obtain an upsampled coarse color set composed of MLPs.

4. The point cloud color upsampling method according to claim 1, characterized in that, The step of using local distance difference as a weight to locate the features of neighboring points and obtaining the geometrically weighted local important color features specifically includes the following steps: The geometric information Copy K times and combine with the grouped geometric information Perform a subtraction operation, process the result using an MLP, and generate distance weights. Based on distance weight and third color feature Multiply to generate the local important color features.

5. The point cloud color upsampling method according to claim 1, characterized in that, The step of refining the coarse color set using the high-dimensional local color features to obtain a fine-grained color set specifically includes the following steps: Based on the high-dimensional local color features Color regression is used, that is, color offset is generated through MLP; The coarse color set Add the color offset to obtain the fine-grained color set. .

6. A system for a point cloud color upsampling method according to claim 1, characterized in that, include: Color information upsampling module: acquires geometric information to upsample sparse color information, and generates a coarse color set through color upsampling; Color information enhancement regression module: Based on the geometric information and deep neural network, extract high-dimensional color features; use local distance difference as weight to locate the features of neighboring points, and obtain the locally important color features weighted by the geometric information; obtain high-dimensional local color features based on the fusion of the high-dimensional color features and the locally important color features; refine the coarse color set through the high-dimensional local color features to obtain a fine-grained color set.

7. A point cloud color upsampling device, characterized in that, The point cloud color upsampling device includes a processor and a memory coupled to the processor, wherein the memory stores program instructions for implementing a point cloud color upsampling method according to any one of claims 1-5; the processor is used to execute the program instructions stored in the memory to implement a point cloud color upsampling.

8. A storage medium, characterized in that, The device stores processor-executable program instructions for performing a point cloud color upsampling method according to any one of claims 1-5.

Images