3D point cloud compression method, apparatus and device, and computer readable storage medium

Abstract

The invention discloses a 3D point cloud compression method, device and equipment and a computer readable storage medium, 3D point cloud data is converted into a distance image, then the distance image is divided into a plurality of blocks, the blocks are distributed to a plurality of planes by using a divide-and-conquer method, each plane only needs to be represented by three parameters, and the compression efficiency is greatly improved. Each plane can represent a plurality of pieces of 3D point cloud data, through the method, the plane with only three parameters is used for representing a plurality of pieces of point cloud, and the storage and transmission quantity is greatly reduced, so that the cost is greatly reduced, and the problem that the existing compression technology possibly cannot meet the requirements of high compression ratio and low delay in medical virtual reality is solved. The technical problem in the prior art is especially solved in scenes requiring real-time processing and transmission of a large amount of data.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a 3D point cloud compression method, device, equipment and computer-readable storage medium. Background Art

[0002] The application of artificial intelligence (AI) in the medical field is becoming more and more extensive, especially in virtual reality (VR) treatment. VR technology provides patients with an immersive experience by simulating the real environment, which shows great potential in psychotherapy, pain management, rehabilitation training, etc.

[0003] In medical virtual reality, 3D point cloud data is usually generated by high-precision scanning equipment, such as CT scans, MRI scans, etc. These data can accurately reflect the patient's body structure and are the key to surgical planning and treatment. In medical virtual reality treatment, 3D point cloud technology can be used to make personalized rehabilitation training plans for patients, and provide a safe exercise environment and real-time feedback. The specific process is as follows:

[0004] (1) Collection and processing of 3D point cloud data: Use RGB-D cameras and other equipment to collect the patient's point cloud data. Preprocess the collected point cloud data to generate a real 3D model.

[0005] (2) Rehabilitation training plan: Develop a personalized rehabilitation training plan based on the patient's physical condition and motor ability.

[0006] (3) Virtual environment establishment: Establish a corresponding virtual training environment based on the formulated training plan.

[0007] (4) Sports training: The patient enters a virtual training environment and completes corresponding sports training.

[0008] (5) Real-time feedback and rehabilitation assessment: The system provides real-time feedback of relevant data and adjusts the training plan based on the patient's exercise status. Regular rehabilitation assessments are conducted on patients to continuously optimize the training plan.

[0009] However, the generation process of 3D point cloud data requires a lot of computing resources, and the amount of data generated is huge, which poses challenges to storage and transmission.

[0010] Currently, KD trees and Octree trees are mostly used to process 3D point cloud data. These data structures are often used for space segmentation and point cloud data compression. However, when processing large-scale point cloud data, their compression rate and efficiency may be insufficient.

[0011] Existing compression technologies may not be able to meet the requirements for high compression rates and low latency in medical VR, especially in scenarios where large amounts of data need to be processed and transmitted in real time. Summary of the invention

[0012] The present application provides a 3D point cloud compression method, apparatus, device and computer-readable storage medium, which solves the technical problem that existing compression technology may not meet the requirements for high compression rate and low latency in medical virtual reality, especially in scenarios where large amounts of data need to be processed and transmitted in real time.

[0013] In view of this, the first aspect of the present application provides a 3D point cloud compression method, the method comprising:

[0014] Step S1: Acquire 3D point cloud data, wherein the 3D point cloud data contains a number of points, and the three-dimensional coordinates of each point are (x i ,y i ,z i );

[0015] Step S2, converting the 3D point cloud data into a two-dimensional distance image;

[0016] Step S3: Based on the divide-and-conquer theory, the distance image is spatially encoded so that each block contained in the distance image is respectively composed of plane coefficients (a * ,b * ,c * ) indicates that compressed data of the 3D point cloud data composed of the plane coefficients is obtained.

[0017] Optionally, the step S2 specifically includes:

[0018] The three-dimensional coordinates of each point of the 3D point cloud data are converted into corresponding pixel points (u, v) contained in the two-dimensional range image by using a preset formula group;

[0019] The preset formula group is specifically:

[0020]

[0021] Where x, y, z represent the three-dimensional coordinates of the point, r represents the distance between the laser instrument and the point cloud sample, W and H represent the number of horizontal and vertical pixels respectively, and the value of W is at least 3 times that of H;

[0022] The distance image Img is:

[0023]

[0024] Among them, I uv represents the pixel value of the uth row and the vth column, and the pixel value indicates whether there is corresponding 3D point cloud data. uv =1 means the corresponding 3D point cloud data exists, otherwise it does not exist.

[0025] Optionally, the step S3 specifically includes:

[0026] Step S31: evenly divide the distance image Img into several blocks of target size, namely Among them, B ij represents the i-th row and j-th column block, M and N represent the number of blocks in the horizontal and vertical directions respectively, and the number is related to the target size;

[0027] Step S32: Block B ij The pixels contained in are fitted into a plane to obtain block B ij The plane coefficient of the corresponding plane (a * ,b * ,c * );

[0028] Step S33: All blocks B ij The plane coefficient of the corresponding plane (a * ,b * ,c * ) constitute a data set, which constitutes the compressed data of the 3D point cloud data.

[0029] Optionally, after step S32 and before step S33, the following steps are further included:

[0030] Step S321, calculate block B ij With block B (i+1)j The fit between

[0031] Step S322: If the degree of fit If it is less than the preset threshold, then set block B (i+1)j The corresponding plane coefficients and block B ij The plane coefficients are consistent, otherwise the block B (i+1)j As block B ij Return and re-execute step S32 until i=M.

[0032] Optionally, after step S32 and before step S33, the following steps are further included:

[0033] Step S321', calculate block B ij With block B i(j+1) The fit between

[0034] Step S322', if the degree of fit If it is less than the preset threshold, then set block B i(j+1) The corresponding plane coefficients and block B ij The plane coefficients are consistent, otherwise the block B i(j+1) As block Bij Return and re-execute step S32 until j=N.

[0035] Optionally, the step S32 specifically includes:

[0036] By solving Get block B ij The plane coefficient of the corresponding plane (a * ,b * ,c * );

[0037] Among them, S is block B ij The number of pixels contained is expressed as represents the L2 norm.

[0038] Optionally, the step S321 or the step S321′ is specifically:

[0039] By calculation Get block B ij With block B i(j+1) The fit between or between blocks B ij With block B (i+1)j The fit between

[0040] Among them, R is block B i(j+1) or block B (i+1)j The number of pixels contained is expressed as

[0041] A second aspect of the present application provides a 3D point cloud compression device, the device comprising:

[0042] The acquisition unit is used to acquire 3D point cloud data, wherein the 3D point cloud data includes a plurality of points, and the three-dimensional coordinates of each point are (x i ,y i ,z i );

[0043] A conversion unit, used for converting the 3D point cloud data into a two-dimensional distance image;

[0044] A compression unit is used to perform spatial encoding on the distance image based on the divide and conquer theory, so that each block included in the distance image is respectively composed of a plane coefficient (a * ,b * ,c * ) indicates that compressed data of the 3D point cloud data composed of the plane coefficients is obtained.

[0045] Optionally, the conversion unit is specifically used for:

[0046] The three-dimensional coordinates of each point of the 3D point cloud data are converted into corresponding pixel points (u, v) contained in the two-dimensional range image by using a preset formula group;

[0047] The preset formula group is specifically:

[0048]

[0049] Where r represents the distance between the laser instrument and the point cloud sample, W and H represent the number of horizontal and vertical pixels respectively, and the value of W is at least 3 times of H;

[0050] The distance image Img is:

[0051]

[0052] Among them, I uv represents the pixel value of the uth row and the vth column, and the pixel value indicates whether there is corresponding 3D point cloud data. uv =1 means the corresponding 3D point cloud data exists, otherwise it does not exist.

[0053] Optionally, the compression unit is specifically used for:

[0054] The distance image Img is evenly divided into several blocks of target size, namely Among them, B ij represents the i-th row and j-th column block, M and N represent the number of blocks in the horizontal and vertical directions respectively, and the number is related to the target size;

[0055] Block B ij The pixels contained in are fitted into a plane to obtain block B ij The plane coefficient of the corresponding plane (a * ,b * ,c * );

[0056] All blocks B ij The plane coefficient of the corresponding plane (a * ,b * ,c * ) constitute a data set, which constitutes the compressed data of the 3D point cloud data.

[0057] Optionally, the compression unit is further used for:

[0058] Calculate block B ij With block B (i+1)j The fit between

[0059] If the fit If it is less than the preset threshold, then set block B(i+1)j The corresponding plane coefficients and block B ij The plane coefficients are consistent, otherwise the block B (i+1)j As block B ij Return and re-execute step S32 until i=M.

[0060] Optionally, the compression unit is further used for:

[0061] Calculate block B ij With block B i(j+1) The fit between

[0062] If the fit If it is less than the preset threshold, then set block B i(j+1) The corresponding plane coefficients and block B ij The plane coefficients are consistent, otherwise the block B i(j+1) As block B ij Return and re-execute step S32 until j=N.

[0063] Optionally, the block B ij The pixels contained in are fitted into a plane to obtain block B ij The plane coefficient of the corresponding plane (a * ,b * ,c * ) specifically include:

[0064] By solving Get block B ij The plane coefficient of the corresponding plane (a * ,b * ,c * );

[0065] Among them, S is block B ij The number of pixels contained is expressed as represents the L2 norm.

[0066] Optionally, the calculation block B ij With block B (i+1)j The fit between Or the calculation block B ij With block B i(j+1) The fit between Specifically:

[0067] By calculation Get block B ij With block B i(j+1) The fit between or between blocks B ij With block B (i+1)j The fit between

[0068] Among them, R is block B i(j+1) or block B (i+1)j The number of pixels contained is expressed as

[0069] A third aspect of the present application provides a 3D point cloud compression device, the device comprising a processor and a memory:

[0070] The memory is used to store program code and transmit the program code to the processor;

[0071] The processor is used to execute the steps of the 3D point cloud compression method as described in the first aspect according to the instructions in the program code.

[0072] A fourth aspect of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store program code, and the program code is used to execute the method described in the first aspect above.

[0073] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0074] In the present application, a 3D point cloud compression method, apparatus, device and computer-readable storage medium are provided, which convert 3D point cloud data into a range image, and then divide the range image into many blocks, and use a divide-and-conquer method to assign these blocks to many planes, each plane only needs 3 parameters to represent, and each plane can represent a lot of 3D point cloud data. Through this method, many point clouds are represented by planes with only 3 parameters, and the number of storage and transmission is greatly reduced, thereby greatly reducing costs, solving the technical problem that existing compression technology may not meet the requirements of high compression rate and low latency in medical virtual reality, especially in scenarios where large amounts of data need to be processed and transmitted in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 This is a method flow chart of the 3D point cloud compression method in an embodiment of the present application;

[0076] Figure 2 This is a schematic diagram of the structure of a 3D point cloud compression device in an embodiment of the present application;

[0077] Figure 3 This is a schematic diagram of the structure of a 3D point cloud compression device in an embodiment of the present application. DETAILED DESCRIPTION

[0078] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0079] The present application designs a 3D point cloud compression method, apparatus, device and computer-readable storage medium, which solves the technical problem that existing compression technology may not meet the requirements for high compression rate and low latency in medical virtual reality, especially in scenarios where large amounts of data need to be processed and transmitted in real time.

[0080] For easier understanding, see Figure 1 , Figure 1 is a method flow chart of the 3D point cloud compression method in the embodiment of the present application, such as Figure 1 As shown, specifically:

[0081] Step S1: Obtain 3D point cloud data. The 3D point cloud data contains several points. The three-dimensional coordinates of each point are (x i ,y i ,z i );

[0082] It should be noted that point cloud data is a set of data consisting of three-dimensional coordinates of points, usually used to represent objects, environments, etc. in three-dimensional scenes. If the 3D point cloud data has n points, n is a natural number, and the three-dimensional coordinates of each point are (x i ,y i ,z i ), then the point cloud data can be represented as an n×3 matrix:

[0083]

[0084] 3D point cloud data is a data structure used to represent objects or scenes in three-dimensional space. It consists of a large number of points, each of which contains its coordinate information in three-dimensional space. These points are usually collected by 3D scanners, laser radar (LIDAR), stereo cameras or other types of sensors. Point cloud data can be used for a variety of applications, including but not limited to:

[0085] 3D modeling: 3D models of objects can be created using point cloud data.

[0086] Robotic Navigation: Environmental perception for robots or self-driving cars.

[0087] Terrain mapping: Used to create three-dimensional maps of terrain.

[0088] Augmented reality: Overlaying virtual information on the real world.

[0089] Cultural Relics Protection: Digital preservation of historical relics.

[0090] Each point in the point cloud data is usually represented by the following information:

[0091] X coordinate: The horizontal position of a point in three-dimensional space.

[0092] Y coordinate: The vertical position of a point in three-dimensional space.

[0093] Z coordinate: the depth position of a point in three-dimensional space.

[0094] In addition to basic 3D coordinates, point cloud data may also contain other information, such as:

[0095] Color information: Each point may contain an RGB color value, indicating the color of the point.

[0096] Normal information: Represents the surface normal of a point, used for rendering and understanding the geometry of an object.

[0097] Intensity information: Some sensors, such as LiDAR, record the intensity of the returning signal, which can be used to distinguish between different materials or surface properties.

[0098] Timestamps: If the point cloud was collected over time, each point may have a timestamp.

[0099] The acquisition and processing of point cloud data is a complex process, involving multiple steps such as data collection, preprocessing (such as denoising, filtering), registration (aligning point cloud data from different perspectives), segmentation (segmenting point clouds into different objects or regions), etc. When processing point cloud data, commonly used software and libraries include PCL (Point Cloud Library), MeshLab, CloudCompare, etc.

[0100] Step S2, converting the 3D point cloud data into a two-dimensional distance image;

[0101] It should be noted that converting 3D point cloud data into distance images is a basic method for extracting 3D information. Compared with the traditional KD tree structure or Octree tree structure, converting into distance images has two major advantages: (1) There is no need to traverse various trees, which is faster. (2) Adjacent pixels in the distance image are likely to belong to the same plane and have spatial locality.

[0102] Converting 3D point cloud data into a 2D range image usually involves the following steps:

[0103] Determine the projection plane: First, you need to determine a plane to project the 3D point cloud onto. This plane can be the XY, YZ, or XZ plane, depending on your needs.

[0104] Project Point Cloud: Projects each point in the point cloud onto the selected plane. For example, if you select the XY plane, then the Z coordinate of each point will be ignored and only the X and Y coordinates will be kept.

[0105] Calculate the distance: For each point on the projection plane, calculate its distance to a reference point (usually the center of the image or a specific point). This distance can be Euclidean distance or other metrics.

[0106] Create a distance image: Use the calculated distance values ​​to create a 2D image. In this image, the brightness or color value of each pixel represents the distance from the 3D point corresponding to that pixel to the reference point. Generally, the farther the distance, the brighter the pixel value or the darker the color.

[0107] Mapping to image resolution: Determine the resolution of the output 2D image and map the point cloud data to the pixels of the image based on this resolution. This may involve interpolation or other image processing techniques.

[0108] Color encoding: If your point cloud data contains color information, you can choose to encode the color information into the 2D image as well, so that each pixel not only represents distance information but also contains color information.

[0109] In practical applications, this process can be implemented through various software and libraries, such as PCL (Point Cloud Library), Open3D, MATLAB, etc. These tools provide a wealth of functions and methods to process point cloud data and generate two-dimensional images.

[0110] Step S3: Based on the divide-and-conquer theory, the distance image is spatially encoded so that each block contained in the distance image is represented by a plane coefficient (a * ,b * ,c * ) indicates that the compressed data of the 3D point cloud data composed of plane coefficients is obtained. * is the normal vector along the a-axis of the lattice, b * is the normal vector along the b-axis of the lattice, c * is the normal vector along the c-axis of the lattice.

[0111] It should be noted that the goal of spatial encoding of the range image is to use a plane with only three parameters to represent multiple sample points, so as to achieve the purpose of compression.

[0112] Spatial encoding of the distance image based on the divide and conquer theory and using the encoded data to represent the compressed data of the 3D point cloud is a method to reduce the amount of data and improve data processing efficiency. This process usually involves the following steps:

[0113] Blocking: Split the range image into multiple small blocks or blocks. Each block contains a portion of the image and can be processed independently. The size of the block can be determined based on the resolution of the image and the desired compression ratio.

[0114] Plane fitting: perform plane fitting on the data points in each block. The purpose of this step is to find an optimal plane so that all point cloud data points in the block can be projected onto this plane as much as possible. A plane is usually represented by the coefficients of a plane equation. For example, the general form in 3D space is ax+by+cz+d=0, where a, b, c are the normal vectors of the plane and d is a constant term.

[0115] Coefficient extraction: Extract plane coefficients from the plane fitting process of each block. These coefficients will be used to describe the local geometric features of the original 3D point cloud data.

[0116] Data compression: Use the extracted plane coefficients to represent the original 3D point cloud data. Since each block can be approximated by a plane, a large amount of point cloud data can be represented by a small amount of data, thus achieving data compression.

[0117] Encoding: Encode the plane coefficients for easy storage or transmission. The encoding can be a simple list of numbers or a more complex data structure, depending on the subsequent application requirements.

[0118] Decoding: When point cloud data is needed, the original 3D point cloud can be restored by decoding the plane coefficients and reconstructing the point cloud data of each block. This process may involve inverse plane fitting for each block to recover the point cloud data from the plane coefficients.

[0119] In practical applications, this process can be implemented using various mathematical and computer vision libraries, such as NumPy, SciPy, Open3D, etc. In addition, the choice of plane fitting algorithm (such as least squares method, RANSAC, etc.) and the design of the encoding scheme will directly affect the quality of compressed data and the accuracy of restored point cloud data.

[0120] Furthermore, step S2 specifically includes:

[0121] The three-dimensional coordinates of each point in the 3D point cloud data are converted into corresponding pixel points (u, v) contained in the two-dimensional range image through a preset formula group;

[0122] The preset formula groups are as follows:

[0123]

[0124] Where r represents the distance between the laser instrument and the point cloud sample, W and H represent the number of horizontal and vertical pixels respectively, and the value of W is at least 3 times of H;

[0125] The distance image Img is:

[0126]

[0127] Among them, I uv Represents the pixel value of the uth row and vth column. The pixel value indicates whether there is corresponding 3D point cloud data. If I uv =1 means the corresponding 3D point cloud data exists, otherwise it does not exist.

[0128] It should be noted that for each pixel point (u,v) of the distance image, it is assumed that the spatial point corresponding to the pixel is (x,y,z). The conversion method from 3D point cloud to 2D distance image is as follows:

[0129]

[0130] Among them, x, y, z represent the three-dimensional coordinates of the point, r represents the distance between the laser instrument and the point cloud sample, W and H represent the horizontal and vertical pixel numbers respectively. Note: W and H can be artificially set hyperparameters. During the setting process, the value of W is at least 3 times that of H.

[0131] Then the distance image Img can be expressed as:

[0132]

[0133] Where W and H represent the width and height respectively. uv Represents the pixel value of the uth row and vth column, which indicates whether there is a corresponding original point cloud. If I uv =1 means the original point cloud sample point exists, otherwise it does not exist.

[0134] A data set consisting of a series of points in three-dimensional space, usually with x, y, and z coordinates representing the position of each point.

[0135] The process of converting to a 2D range image involves converting the coordinates of each 3D point into pixels on a 2D image. This 2D image is called a range image, and it uses pixel values ​​to represent the presence or absence of each point in the 3D point cloud.

[0136] The above conversion is performed through the preset formula group, and the meanings of the formulas contained in it are:

[0137] To calculate the Euclidean distance from each point in the point cloud to the origin (usually the location of the laser scanner);

[0138] To calculate the pixel position corresponding to the point in the horizontal direction, the inverse tangent function is used to calculate the angle and normalized by dividing by W (the number of pixels in the vertical direction);

[0139] To calculate the pixel position corresponding to the point in the vertical direction, the arc cosine function is used here to calculate the angle and normalize it by dividing it by H (the number of pixels in the horizontal direction).

[0140] For the distance image Img, this is a two-dimensional array, where each element I uv Represents a pixel value. If I uv =1, it means that there is point cloud data in the corresponding 3D space; if I uv =0, it means it does not exist.

[0141] In a range image, the pixel value is usually a binary flag indicating whether there is 3D point cloud data corresponding to the position of that pixel. This representation method simplifies the data so that each pixel only contains information about whether a point exists or not, rather than the exact position or distance of the point.

[0142] In the embodiment of the present application, the value of W is at least 3 times the coefficient of H, but this coefficient can be adjusted in actual applications according to specific applications or scanning equipment, as long as the value of W is much larger than H.

[0143] Through this process, the raw 3D point cloud data is converted into a 2D image that can be processed and analyzed in a simpler way, such as in computer vision and machine learning applications. This conversion also helps reduce the complexity of the data, making it easier to store and transmit.

[0144] Furthermore, step S3 specifically includes:

[0145] Step S31: evenly divide the distance image Img into several blocks of target size, namely Among them, B ij represents the i-th row and j-th column block, M and N represent the number of blocks in the horizontal and vertical directions respectively, and the number is related to the target size;

[0146] It should be noted that the distance image Img obtained in the previous step is evenly divided into many blocks. If the size of each block is 4×4, it can be expressed as

[0147]

[0148] Among them B ij represents the front of the i-th row and j-th column, because the size of each block is 4×4, then M and N represent the number of blocks in the horizontal and vertical directions respectively.

[0149] Step S32: Block B ij The pixels contained in are fitted into a plane to obtain block B ij The plane coefficient of the corresponding plane (a * ,b * ,c * );

[0150] It should be noted that in a 3D Cartesian space, a plane can be represented as:

[0151] x+ay+bz-c=0

[0152] Where (1, a, b) is the normal vector of a plane, is the distance from the laser radar's emission point to the plane. Then, all sample points falling on this plane only need these three coefficients (a, b, c) to represent them, which greatly reduces the amount of data storage and transmission.

[0153] If B ij This block has S sample points, expressed as Then use all the points of this block to fit a plane. The formula is as follows:

[0154]

[0155] in represents the L2 norm. By solving this formula, we can get the plane coefficient we want (a * ,b * ,c * ).

[0156] Step S33: All blocks B ij The plane coefficient of the corresponding plane (a * ,b * ,c * ) form a data set, which constitutes the compressed data of 3D point cloud data.

[0157] Furthermore, after step S32 and before step S33, the following steps are further included:

[0158] Step S321, calculate block B ij With block B (i+1)j The fit between

[0159] Step S322: If the degree of fit If it is less than the preset threshold, then set block B (i+1)j The corresponding plane coefficients and block B ij The plane coefficients are consistent, otherwise the block B (i+1)j As block B ij Return and re-execute step S32 until i=M.

[0160] Furthermore, after step S32 and before step S33, the following steps are further included:

[0161] Step S321', calculate block B ij With block B i(j+1) The fit between

[0162] Step S322', if the degree of fit If it is less than the preset threshold, then set block B i(j+1) The corresponding plane coefficients and block B ij The plane coefficients are consistent, otherwise the block B i(j+1) As block B ij Return and re-execute step S32 until j=N.

[0163] It should be noted that, for the fitted plane, there may be a situation where the adjacent block has a high degree of fit with the plane, and the plane can also be used to represent the block.

[0164] If the next block B i(j+1) or block B (i+1)j There are R sample points, expressed as The goodness of fit can be expressed using distance:

[0165]

[0166] Where (a * ,b * ,c * ) is block B ij Calculated plane coefficient.

[0167] Each block is assigned to each plane according to the divide-and-conquer strategy. If the fit degree φ of all points in the next block with the current plane is less than the threshold set manually, then all points in this block are also represented by this plane, otherwise, the next plane is used. This can reduce the amount of data because the same plane equation can be used to describe two blocks.

[0168] This process is an application of the divide and conquer strategy, achieving global consistency and optimization through local optimization. This method is particularly useful when processing large-scale 3D point cloud data, which can effectively reduce the complexity of the data while maintaining the geometric characteristics of the data. This technology is widely used in 3D modeling, virtual reality, game development, and computer-aided design.

[0169] When processing 3D point cloud data, the calculation of fit is a key step to measure the degree of match between a plane model and the actual data point set. In addition to the distance-based fit representation scheme mentioned above, the following different fit representation methods can also be considered to adapt to different application scenarios and improve the accuracy of the model:

[0170] Average distance error:

[0171] Calculate the distance from each point to the fitting plane and then take the average value. The smaller the value, the better the fit between the point cloud and the plane.

[0172] Maximum distance error:

[0173] The maximum distance of all points to the fitted plane is calculated, which ensures that even the farthest points have a good fit to the plane.

[0174] Root Mean Square Error (RMSE):

[0175] Similar to the method already mentioned, but taking the square root, which can reduce the impact of outliers.

[0176] Standard Deviation:

[0177] Calculate the standard deviation of the distance from the point to the plane, which helps to understand how scattered the data points are.

[0178] Correlation coefficient:

[0179] The correlation coefficient between the coordinates of the point cloud data and the plane equation is used to measure the fit. The closer the correlation coefficient is to 1 or -1, the better the fit.

[0180] Goodness of fit test:

[0181] Use statistical methods, such as the coefficient of determination (R-squared), to measure how well the model explains the data.

[0182] Area-based weighting:

[0183] Different regions (such as points close to the center of the plane) can be given different weights to reflect their importance in the fitting.

[0184] These methods can be selected and adjusted according to specific application requirements and data characteristics to achieve the best fitting effect. In practical applications, it may be necessary to combine multiple methods to comprehensively evaluate the fit to ensure the accuracy and robustness of the model.

[0185] Further, step S32 specifically includes:

[0186] By solving Get block B ij The plane coefficient of the corresponding plane (a * ,b * ,c * );

[0187] Among them, S is block B ij The number of pixels contained is expressed as represents the L2 norm.

[0188] It should be noted that a plane is fitted by a mathematical optimization method, and the plane can best represent a block in the point cloud data. Specifically, step S32 involves the following process:

[0189] Optimization goal:

[0190] Find a plane that minimizes the total square error between the plane and all point cloud data points in the block. This plane can be represented by the equation ax+by+cz+d=0, where a, b, c are the normal vectors of the plane and d is a constant term.

[0191] Mathematical expression:

[0192] The optimization problem can be expressed as Among them, (a * ,b * ,c * ) is block B ij The three-dimensional coordinates of the s-th point in the block, S is the total number of points in the block, Represents the L2 norm, which is the square of the Euclidean distance, and is used to calculate the distance from a point to a plane.

[0193] By minimizing the above equation, we can solve the plane coefficient (a * ,b * ,c * ), these coefficients define the best fit plane.

[0194] This process usually involves linear algebra and numerical optimization techniques such as least squares methods.

[0195] Least Squares Method:

[0196] The method of least squares is a common mathematical technique used to find the best fitting line or plane given a set of data points.

[0197] In this case, the least squares method will calculate a plane that minimizes the sum of the squares of the perpendicular distances of all points to the plane.

[0198] application:

[0199] After finding the best fitting plane, the equation of this plane can be used to approximate all the point cloud data points in the block.

[0200] This method can significantly reduce the amount of data, because a plane only needs three parameters (a * ,b * ,c * ) instead of three coordinates for each point.

[0201] Further, step S321 or step S321' is specifically:

[0202] By calculation Get block B ij With block B i(j+1) The fit between or between blocks B ij With block B (i+1)j The fit between

[0203] Among them, R is block B i(j+1) or block B (i+1)j The number of pixels contained is expressed as

[0204] It should be noted that by calculating the degree of fit between two adjacent blocks, it is determined whether the two adjacent blocks are geometrically close enough to be represented by the same plane.

[0205] The above formula is used to calculate the degree of fit φ, which measures the sum of the squares of the distances from the points in the block to a plane equation. The smaller the distance, the closer the points in the block are to the plane, and the higher the degree of fit.

[0206] For each adjacent block, calculate the above formula to get a goodness of fit value;

[0207] This value reflects the difference between the point in the block and the block B. ij The overall deviation of the plane equation. ;

[0208] The calculated fit value is compared with the preset threshold. If the fit value is less than the threshold, it means that the adjacent block and block B are ijThe plane fits well, so they can be considered geometrically consistent, so the same plane coefficient can be used to represent the two blocks; otherwise, it is considered that the adjacent block is consistent with block B. ij The plane fit is not good, and a new plane coefficient needs to be found or calculated for the adjacent block.

[0209] Through this method, the number of planes used to represent 3D point cloud data can be effectively reduced, thereby reducing the amount of data and achieving data compression.

[0210] Optimizing the goodness-of-fit calculation process to improve efficiency is a key issue in 3D point cloud data processing. The following are some strategies and techniques that can help improve the efficiency of goodness-of-fit calculation:

[0211] Use efficient numerical methods:

[0212] Use efficient numerical optimization algorithms, such as gradient descent, conjugate gradient method, or Levenberg-Marquardt algorithm, to solve least squares problems.

[0213] For large-scale problems, you can consider using sparse matrix technology and parallel computing to speed up the calculation process.

[0214] Reduce data points:

[0215] Before calculating the fit, the point cloud data is downsampled or downsampled to reduce the number of data points involved in the calculation.

[0216] Use methods such as FarthestPoint Sampling or random sampling to select a representative point set.

[0217] Block strategy:

[0218] Divide the point cloud data into smaller blocks and calculate the fit independently for each small block, which can be processed in parallel and reduce the amount of calculation.

[0219] Approximate method:

[0220] Use an approximate method to estimate the goodness of fit, for example, by randomly selecting a subset of points to approximate the goodness of fit for the entire patch.

[0221] Cache and reuse results:

[0222] For neighboring tiles, their geometric characteristics are likely to be similar, and the calculated plane coefficients can be cached and the results reused when possible.

[0223] Use a precomputed model:

[0224] For some applications, some common geometric models can be pre-computed and used directly in the actual calculations instead of fitting from scratch.

[0225] parallel computing:

[0226] By utilizing the multi-core and multi-processor architecture of modern computers, the fitness calculation task is distributed to multiple cores for parallel processing.

[0227] Hardware Acceleration:

[0228] Numerical computations are performed using GPUs or other specialized hardware accelerators that are more efficient at handling large-scale parallel computing tasks.

[0229] Optimization algorithm implementation:

[0230] Optimize algorithm code, for example, reduce unnecessary data copying, use efficient data structures and algorithm libraries.

[0231] Threshold selection:

[0232] Reasonable selection of the fitting threshold and avoidance of overly strict fitting requirements can reduce the number of blocks that require fine fitting.

[0233] Using Machine Learning:

[0234] Machine learning models are trained to predict the fit of the chunks, which allows new data to be classified and processed quickly.

[0235] Multi-scale approach:

[0236] The fit is calculated at different scales, starting with a quick fit at a coarser scale and then a finer fit in areas where it is needed.

[0237] Through the above method, the efficiency of the fit calculation can be significantly improved while maintaining the accuracy of the calculation results. In practical applications, it may be necessary to select appropriate strategies and techniques based on specific data characteristics and computing resources.

[0238] Specifically, input: distance image X, the size of the distance image is W×H, output: a plane set The spatial coding method based on "divide and conquer" provided in this application is as follows:

[0239] [1] Divide the distance image Img evenly into many blocks. If the size of each block is 4×4, it can be expressed as:

[0240]

[0241] Among them B ijrepresents the front of the i-th row and j-th column, because the size of each block is 4×4, then M and N represent the number of blocks in the horizontal and vertical directions respectively.

[0242] 【2】Initialize the first plane to be empty P k ={0}where k=1.

[0243] Fori←1…M

[0244] Forj←1…N

[0245] IF P k =={0}

[0246] 【3】Use the points of the current block to fit a plane, and the fitting degree is set to

[0247] Else

[0248] 【4】Current block B ij The fit between all point cloud sample points and the current plane

[0249] EndIF

[0250] IF / / Threshold is a manually set threshold

[0251] 【5】Point B of the current block ij All use current plane P k express

[0252] Else

[0253] 【6】Take out the next plane and initialize it to zero k=k+1 and P k ={0}

[0254] EndIF

[0255] End

[0256] EndFor

[0257] The 3D point cloud compression technology provided by this application can be widely used in multiple fields, especially in scenarios that require real-time processing and transmission of large amounts of three-dimensional data, such as the financial field, medical virtual reality (VR), autonomous driving, robot navigation, digital protection of cultural relics, etc. Among them, in the financial field, 3D point cloud data can be used for a variety of applications, such as building information modeling (BIM) in real estate appraisal, data analysis of security monitoring systems, customer behavior analysis, etc. The following is a specific application example to illustrate the application of the technology of this application in the financial field:

[0258] Application scenario: real estate investment analysis

[0259] background:

[0260] In the field of financial investment, accurate evaluation of real estate is crucial. 3D point cloud data can provide accurate three-dimensional models of buildings, helping investors make more accurate analysis and decisions.

[0261] Application Process:

[0262] Data collection:

[0263] Use the laser scanner carried by the drone to scan the target building and obtain the 3D point cloud data of the building.

[0264] Data preprocessing:

[0265] The collected point cloud data is pre-processed by denoising, filtering and other operations to improve the data quality.

[0266] Point Cloud Compression:

[0267] The 3D point cloud compression method provided in this application is used to convert the preprocessed point cloud data into a two-dimensional distance image.

[0268] The distance image is divided into multiple small blocks, and a plane fitting is performed on each small block to obtain the plane coefficient.

[0269] Use plane coefficients to compress point cloud data and significantly reduce the amount of data.

[0270] Real estate appraisal:

[0271] The compressed point cloud data is used to reconstruct the 3D model of the building in professional analysis software.

[0272] Analysis software can calculate key indicators such as building area, structural integrity, material consumption, etc.

[0273] Investment decision support:

[0274] Investors use 3D models and analysis results to make investment decisions and assess the value and potential risks of real estate.

[0275] Through virtual models, investors can conduct virtual inspections and understand the property conditions without having to visit the property in person.

[0276] Data Sharing and Reporting:

[0277] The compressed point cloud data can be easily shared with remote investors and analysts via the Internet.

[0278] The generated assessment report can include an interactive 3D view to improve the professionalism and intuitiveness of the report.

[0279] Technical advantages:

[0280] Accurate assessment: 3D point cloud data allows for more accurate real estate assessments and reduces human error.

[0281] Improved efficiency: Compression technology reduces the time for data transmission and processing, thus improving the efficiency of evaluation work.

[0282] Cost savings: Reduces the number and cost of field visits, and also reduces the cost of data storage.

[0283] Remote collaboration: Investors and analysts can access and analyze data remotely, increasing the flexibility of collaboration.

[0284] result:

[0285] Through the 3D point cloud compression technology of this application, financial institutions can conduct real estate investment analysis more quickly and accurately, improving the quality and efficiency of decision-making. At the same time, investors can make smarter investment choices based on more comprehensive data and reduce investment risks.

[0286] See also Figure 2 , Figure 2 Schematic diagram of the structure of the 3D point cloud compression device in the embodiment of the present application. Figure 2 As shown, specifically:

[0287] The acquisition unit 201 is used to acquire 3D point cloud data. The 3D point cloud data includes a number of points. The three-dimensional coordinates of each point are (x i ,y i ,z i );

[0288] A conversion unit 202, used to convert the 3D point cloud data into a two-dimensional range image;

[0289] The compression unit 203 is used to perform spatial encoding on the distance image based on the divide and conquer theory, so that each block included in the distance image is respectively composed of plane coefficients (a * ,b * ,c * ) indicates that the compressed data of the 3D point cloud data composed of plane coefficients is obtained.

[0290] Another embodiment of the present invention provides a 3D point cloud compression device, such as Figure 3 As shown, the device 10 includes:

[0291] One or more processors 110 and memory 120, Figure 3A processor 110 is used as an example for description. The processor 110 and the memory 120 may be connected via a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0292] The processor 110 is used to complete various control logics of the device 10, and it can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a single-chip microcomputer, an ARM (Acorn RISCMachine) or other programmable logic device, a discrete gate or transistor logic, a discrete hardware component, or any combination of these components. In addition, the processor 110 can also be any traditional processor, microprocessor or state machine. The processor 110 can also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors combined with a DSP and / or any other such configuration.

[0293] The memory 120 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions corresponding to the method for constructing a multilingual phoneme representation model in the embodiment of the present invention. The processor 110 executes various functional applications and data processing of the device 10 by running the non-volatile software programs, instructions and units stored in the memory 120, that is, implements the method for constructing a multilingual phoneme representation model in the above method embodiment.

[0294] The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the device 10, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 120 may optionally include a memory remotely arranged relative to the processor 110, and these remote memories may be connected to the device 10 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0295] One or more units are stored in the memory 120, and when executed by one or more processors 110, the following steps are implemented:

[0296] Get 3D point cloud data, the 3D point cloud data contains a number of points, each of which has a 3D coordinate (x i ,y i ,z i );

[0297] Converting the 3D point cloud data into a two-dimensional range image;

[0298] Based on the divide and conquer theory, the distance image is spatially encoded so that each block contained in the distance image is composed of plane coefficients (a * ,b * ,c * ) indicates that compressed data of the 3D point cloud data composed of the plane coefficients is obtained.

[0299] An embodiment of the present invention provides a non-volatile computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by one or more processors, any one of the implementation methods of a 3D point cloud compression method described in the above embodiments is implemented.

[0300] As an example, non-volatile storage media can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) as external cache memory. By way of illustration and not limitation, RAM can be obtained in many forms such as synchronous RAM (SRAM), dynamic RAM, (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), and direct Rambus RAM (DRRAM). The disclosed memory components or memories of the operating environment described herein are intended to include one or more of these and / or any other suitable types of memory.

[0301] In an embodiment of the present application, a 3D point cloud compression method, apparatus, device and computer-readable storage medium are provided, which convert 3D point cloud data into a range image, and then divide the range image into many blocks. These blocks are assigned to many planes using a divide-and-conquer method. Each plane only needs 3 parameters to represent it, and each plane can represent a lot of 3D point cloud data. Through this method, many point clouds are represented by planes with only 3 parameters, and the amount of storage and transmission is greatly reduced, thereby greatly reducing costs. This solves the technical problem that existing compression technologies may not be able to meet the requirements of high compression rate and low latency in medical virtual reality, especially in scenarios where large amounts of data need to be processed and transmitted in real time.

[0302] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0303] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein, for example. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0304] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0305] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0306] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0307] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0308] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (full name in English: Read-Only Memory, English abbreviation: ROM), random access memory (full name in English: Random Access Memory, English abbreviation: RAM), disk or optical disk and other media that can store program codes.

[0309] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

[0310] It should be noted that if software tools or components other than those of our company appear in the embodiments of the present application, they are only used for illustrative purposes and do not represent actual use.

Claims

1. A 3D point cloud compression method, characterized in that: include: Step S1: Acquire 3D point cloud data, wherein the 3D point cloud data contains a number of points, and the three-dimensional coordinates of each point are (x i ,y i ,z i ); Step S2, converting the 3D point cloud data into a two-dimensional distance image; Step S3: Based on the divide-and-conquer theory, the distance image is spatially encoded so that each block contained in the distance image is respectively composed of plane coefficients (a * ,b * ,c * ) indicates that compressed data of the 3D point cloud data composed of the plane coefficients is obtained.

2. The 3D point cloud compression method according to claim 1, characterized in that: The step S2 specifically includes: The three-dimensional coordinates of each point of the 3D point cloud data are converted into corresponding pixel points (u, v) contained in the two-dimensional range image by using a preset formula group; The preset formula group is specifically: Where x, y, z represent the three-dimensional coordinates of the point, r represents the distance between the laser instrument and the point cloud sample, W and H represent the number of horizontal and vertical pixels respectively, and the value of W is at least 3 times that of H; The distance image Img is: Among them, I uv represents the pixel value of the uth row and the vth column, and the pixel value indicates whether there is corresponding 3D point cloud data. uv =1 means the corresponding 3D point cloud data exists, otherwise it does not exist.

3. The 3D point cloud compression method according to claim 2, characterized in that: The step S3 specifically includes: Step S31: evenly divide the distance image Img into several blocks of target size, namely Among them, B ij represents the i-th row and j-th column block, M and N represent the number of blocks in the horizontal and vertical directions respectively, and the number is related to the target size; Step S32: Block B ij The pixels contained in are fitted into a plane to obtain block B ij The plane coefficient of the corresponding plane (a * ,b * ,c * ); Step S33: All blocks B ij The plane coefficient of the corresponding plane (a * ,b * ,c * ) constitute a data set, which constitutes the compressed data of the 3D point cloud data.

4. The 3D point cloud compression method according to claim 3, characterized in that: After step S32 and before step S33, the following steps are also included: Step S321, calculate block B ij With block B (i+1)j The fit between Step S322: If the degree of fit If it is less than the preset threshold, then set block B (i+1)j The corresponding plane coefficients and block B ij The plane coefficients are consistent, otherwise the block B (i+1)j As block B ij Return and re-execute step S32 until i=M.

5. The 3D point cloud compression method according to claim 3, characterized in that: After step S32 and before step S33, the following steps are also included: Step S321', calculate block B ij With block B i(j+1) The fit between Step S322', if the degree of fit If it is less than the preset threshold, then set block B i(j+1) The corresponding plane coefficients and block B ij The plane coefficients are consistent, otherwise the block B i(j+1) As block B ij Return and re-execute step S32 until j=N.

6. The 3D point cloud compression method according to claim 3, characterized in that: The step S32 specifically includes: By solving Get block B ij The plane coefficient of the corresponding plane (a * ,b * ,c * ); Among them, S is block B ij The number of pixels contained is expressed as represents the L2 norm.

7. The 3D point cloud compression method according to claim 4 or 5, characterized in that: The step S321 or the step S321' is specifically as follows: By calculation Get block B ij With block B i(j+1) The fit between or between blocks B ij With block B (i+1)j The fit between Among them, R is block B i(j+1) or block B (i+1)j The number of pixels contained is expressed as 8. A 3D point cloud compression device, characterized in that: include: The acquisition unit is used to acquire 3D point cloud data, wherein the 3D point cloud data includes a plurality of points, and the three-dimensional coordinates of each point are (x i ,y i ,z i ); A conversion unit, used for converting the 3D point cloud data into a two-dimensional distance image; A compression unit is used to perform spatial encoding on the distance image based on the divide and conquer theory, so that each block included in the distance image is respectively composed of a plane coefficient (a * ,b * ,c * ) indicates that compressed data of the 3D point cloud data composed of the plane coefficients is obtained.

9. A 3D point cloud compression device, characterized in that: The device comprises a processor and a memory: The memory is used to store program code and transmit the program code to the processor; The processor is used to execute the 3D point cloud compression method according to any one of claims 1-7 according to the instructions in the program code.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program code, and the program code is used to execute the 3D point cloud compression method according to any one of claims 1 to 7.

Images