Point cloud downsampling method and device

Abstract

The invention discloses a point cloud downsampling method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the following steps: determining a plurality of target points in a target point cloud; for each target point, determining at least one first neighbor point of the target point from the target point cloud; determining an inverse density value of the target point according to the distance between the target point and each first neighbor point; selecting a plurality of central points from the plurality of target points according to the inverse density value of each target point; and performing down-sampling processing on the target point cloud according to the plurality of center points. According to the embodiment, the loss of the key point data after down-sampling can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a point cloud downsampling method and device. Background Art

[0002] The point data set obtained by scanning the surface of an object using a 3D laser scanner is called a point cloud. The number of point data in a point cloud is relatively large and dense. Therefore, downsampling the point cloud is a key step in processing the point cloud. Currently, the point cloud is mainly downsampled by random sampling. However, the random sampling method cannot capture the key data of the entire object, which will cause the loss of some key point data. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a point cloud down-sampling method and device, which can reduce the loss of key point data after down-sampling.

[0004] In a first aspect, an embodiment of the present invention provides a point cloud downsampling method, comprising:

[0005] Determine multiple target points in the target point cloud;

[0006] For each of the target points, determining at least one first neighbor point of the target point from the target point cloud; determining an inverse density value of the target point according to the distance between the target point and each of the first neighbor points;

[0007] Selecting a plurality of center points from the plurality of target points according to the inverse density value of each of the target points;

[0008] The target point cloud is downsampled according to the multiple center points.

[0009] Optionally, performing downsampling processing on the target point cloud according to the multiple center points includes:

[0010] Determine a current center point from the multiple center points;

[0011] Determine at least one second neighbor point of the current center point from the multiple center points;

[0012] Determine enhanced feature information of each of the second neighbor points by using a self-attention mechanism;

[0013] According to the enhanced feature information of each of the second neighboring points, the coordinate information and the original feature information of the current center point are updated;

[0014] The updated current center point is determined as a down-sampling point corresponding to the target point cloud, so as to perform down-sampling processing on the target point cloud.

[0015] Optionally, the using a self-attention mechanism to determine enhanced feature information of each second neighbor point includes:

[0016] Constructing a neighbor matrix according to the original feature information of each of the second neighbor points;

[0017] Input the neighbor matrix into the self-attention model to obtain a query matrix, a key matrix and a value matrix;

[0018] Generate an attention matrix according to the query matrix and the key matrix;

[0019] Generate a self-attention score according to the attention matrix and the value matrix;

[0020] According to the self-attention score and the original feature information of the second neighbor point, enhanced feature information of the second neighbor point is obtained.

[0021] Optionally, constructing a neighbor matrix according to original feature information of each of the second neighbor points includes:

[0022] Determine relative coordinate information of the second neighbor point relative to the current center point;

[0023] splicing the relative coordinate information and the original feature information of the second neighbor point to obtain splicing feature information of the second neighbor point;

[0024] The splicing feature information of each of the second neighbor points is combined to generate the neighbor matrix.

[0025] Optionally, the updating of the coordinate information and original feature information of the current center point according to the enhanced feature information of each of the second neighboring points includes:

[0026] Constructing an enhanced neighbor matrix according to the enhanced feature information of each of the second neighbor points;

[0027] Inputting the enhanced neighbor matrix into a first perception model to obtain a first output matrix;

[0028] Normalizing the first output matrix to generate a coordinate update matrix;

[0029] The coordinate information of the current center point is updated according to the coordinate update matrix.

[0030] Optionally, the updating of the coordinate information and original feature information of the current center point according to the enhanced feature information of each of the second neighboring points includes:

[0031] Constructing an enhanced neighbor matrix according to the enhanced feature information of each of the second neighbor points;

[0032] Inputting the enhanced neighbor matrix into a second perception model to obtain a second output matrix;

[0033] Normalizing the second output matrix to generate a feature update matrix;

[0034] The feature information of the current center point is updated according to the feature update matrix.

[0035] Optionally, performing downsampling processing on the target point cloud according to the multiple center points includes:

[0036] The multiple center points are used as down-sampling points corresponding to the target point cloud, and down-sampling processing is performed on the target point cloud.

[0037] In a second aspect, an embodiment of the present invention provides a point cloud downsampling device, comprising:

[0038] A point determination module, used to determine multiple target points in the target point cloud;

[0039] a value determination module, configured to determine, for each of the target points, at least one first neighbor point of the target point from the target point cloud; and determine an inverse density value of the target point according to a distance between the target point and each of the first neighbor points;

[0040] A point selection module, configured to select a plurality of center points from the plurality of target points according to the inverse density value of each of the target points;

[0041] A downsampling module is used to downsample the target point cloud according to the multiple center points.

[0042] In a third aspect, an embodiment of the present invention provides an electronic device, including:

[0043] one or more processors;

[0044] a storage device for storing one or more programs,

[0045] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above embodiments.

[0046] In a fourth aspect, an embodiment of the present invention provides a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the above embodiments.

[0047] One embodiment of the above invention has the following advantages or beneficial effects: the inverse density value of the target point is determined according to the distance between the target point and each of the first neighboring points of the target point. The inverse density value is a statistical value of the distance between the target point and each of the first neighboring points, such as the sum, average value, mode, etc. The inverse density value can reflect the distribution of each target point in the target point cloud. Compared with the random sampling method, the scheme of the embodiment of the present invention selects multiple center points through the inverse density value to perform down-sampling processing on the target point cloud, which can select the down-sampling points more evenly, retain important information in the point cloud, and reduce the loss of key point data after down-sampling.

[0048] In addition, determining the first neighbor points of the target point and calculating the distance between the target point and the first neighbor points are usually necessary steps for processing point clouds. Therefore, the solution of the embodiment of the present invention can be directly implemented using the calculated distances between the target point and the first neighbor points, which has a high execution efficiency.

[0049] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0051] Figure 1 is a schematic diagram of a process of a point cloud downsampling method provided by an embodiment of the present invention;

[0052] Figure 2 is a comparison diagram of sampling results of a random sampling method and a point cloud downsampling method provided by an embodiment of the present invention;

[0053] Figure 3 is a schematic diagram of a process of a point cloud downsampling method provided by another embodiment of the present invention;

[0054] Figure 4 is a schematic diagram of a process for enhancing feature information of neighboring points by using a self-attention mechanism, provided by an embodiment of the present invention;

[0055] Figure 5 is a schematic diagram of updating the coordinate information and feature information of a center point by using enhanced neighboring points provided by an embodiment of the present invention;

[0056] Figure 6a It is a schematic diagram of comparing the implementation effects of a downsampling algorithm provided by an embodiment of the present invention;

[0057] Figure 6b is a schematic diagram of another downsampling algorithm implementation effect comparison provided by an embodiment of the present invention;

[0058] Figure 7a It is a line chart of robustness verification of multiple algorithms provided by an embodiment of the present invention;

[0059] Figure 7b It is another line chart of robustness verification of multiple algorithms provided by an embodiment of the present invention;

[0060] Figure 8 is a structural schematic diagram of a point cloud downsampling device provided by an embodiment of the present invention;

[0061] Fig. 9 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0063] It should be noted that the acquisition, storage, use, and processing of data in the technical solutions of the embodiments of the present invention are in compliance with the relevant provisions of national laws and regulations.

[0064] Figure 1 FIG. 1 is a schematic diagram of a process of a point cloud downsampling method provided by an embodiment of the present invention. Figure 1 As shown, the method includes:

[0065] Step 101: Determine a plurality of target points in a target point cloud.

[0066] All points in the target point cloud can be used as target points, or multiple target points can be randomly determined from the target point cloud.

[0067] Step 102: for each target point, determine at least one first neighbor point of the target point from the target point cloud; and determine the inverse density value of the target point according to the distance between the target point and each first neighbor point.

[0068] The first neighbor point is the point in the target point cloud that is the closest neighbor to the target point. The number of first neighbor points can be set according to specific needs. The distance between two points can be the Euclidean distance, Manhattan distance, Chebyshev distance, etc. The inverse density value is the statistical value of the distance between the target point and each first neighbor point. The statistical value can be the sum of the distance between the target point and each first neighbor point, the average value of the distance between the target point and each first neighbor point, the mode of the distance between the target point and each first neighbor point, etc.

[0069] In one embodiment of the present invention, the inverse density value can be calculated by the following formula:

[0070]

[0071] In the above formula, φ(I) represents the inverse density function of point I, ′ Represents the first neighbor point of target point I, N k(I) is the set of first neighbor points of the target point. The sum of the distances between the target point and the k first neighbor points is the inverse density value of I. Compared with the random sampling algorithm, this method can select the center point more evenly through coarse resolution to retain important information.

[0072] Step 103: Select multiple center points from multiple target points according to the inverse density value of each target point.

[0073] According to step 102, the inverse density value of each target point in the target point cloud is calculated; the target points are sorted in descending order according to the inverse density value; the first N target points with the largest inverse density value are taken as the center point, where N is a positive integer. In addition, multiple target points with inverse density values ​​greater than the density threshold can also be taken as the center point.

[0074] Step 104: Down-sampling the target point cloud according to the multiple center points.

[0075] Multiple center points can be directly used as downsampling points corresponding to the target point cloud to perform downsampling processing on the target point cloud. Since there may be noise points in the target point cloud, multiple center points can also be denoised, and the denoised center points are used as downsampling points to perform downsampling processing on the target point cloud to improve the sampling effect of downsampling.

[0076] In an embodiment of the present invention, the inverse density value of the target point is determined according to the distance between the target point and each of the first neighboring points of the target point. The inverse density value is a statistical value of the distance between the target point and each of the first neighboring points, such as the sum, average value, mode, etc. The inverse density value can reflect the distribution of each target point in the target point cloud. Compared with the random sampling method, the scheme of the embodiment of the present invention selects multiple center points through the inverse density value to perform down-sampling processing on the target point cloud, which can select the down-sampling points more evenly, retain important information in the point cloud, and reduce the loss of key point data after down-sampling.

[0077] In addition, determining the first neighbor points of the target point and calculating the distance between the target point and the first neighbor points are usually necessary steps for processing point clouds. Therefore, the solution of the embodiment of the present invention can be directly implemented using the calculated distances between the target point and the first neighbor points, which has a high execution efficiency.

[0078] Figure 2 is a comparison chart of the sampling results of the random sampling method and the downsampling method provided by the embodiment of the present invention. Figure 2 As shown, the three pictures from left to right are: the result picture using the random sampling method, the result picture using the downsampling method provided by the embodiment of the present invention, and the original image. For a simple chair model, the legs and supporting hands in the chair model correspond to a sparse set of points. Random sampling will obviously lose important feature information, such as the legs and supporting hands of the chair. The downsampling method provided by the embodiment of the present invention retains the key points of the legs and supporting hands of the chair, which is much better than the random sampling method. It can be seen that the effect of the embodiment of the present invention is much more uniform than random sampling.

[0079] Figure 1 The downsampling method shown will give priority to sampling noise points in the point cloud. If the center point is directly used as the downsampling point of the target point cloud, the downsampling effect will be affected. The embodiment of the present invention also provides another point cloud downsampling method. Figure 3 FIG. 1 is a schematic diagram of a process of a point cloud downsampling method provided by another embodiment of the present invention. Figure 3 As shown, the method includes:

[0080] Step 301: Determine a plurality of target points in a target point cloud.

[0081] Step 302: for each target point, determine at least one first neighbor point of the target point from the target point cloud; and determine the inverse density value of the target point according to the distance between the target point and each first neighbor point.

[0082] Step 303: Select multiple center points from multiple target points according to the inverse density value of each target point.

[0083] Step 304: Determine the current center point from the multiple center points.

[0084] Step 305: Determine at least one second neighbor point of the current center point from the multiple center points.

[0085] The second neighbor point is the point that is closest to the current center point among the multiple center points. The number of the second neighbor points can be set according to specific needs.

[0086] Step 306: Determine enhanced feature information of each second neighbor point using a self-attention mechanism.

[0087] The self-attention mechanism is used to enhance the original feature information of each second neighbor point so that the enhanced feature information of the second neighbor point conforms to the data scene in the target point cloud.

[0088] Step 307: Update the coordinate information and original feature information of the current center point according to the enhanced feature information of each second neighbor point.

[0089] If the current center point is a noise point, the current center point is updated to adjust the coordinate information and feature information of the current center point, so that the updated current center point is close to the real data in the target point cloud. If the current center point is not a noise point, the current center point is updated using the attention mechanism, which will not cause significant changes in the coordinate information and feature information of the current center point.

[0090] Steps 304 to 307 are executed repeatedly until all target points are updated.

[0091] Step 308: Determine the updated current center point as the down-sampling point corresponding to the target point cloud, so as to perform down-sampling processing on the target point cloud.

[0092] In the embodiment of the present invention, the attention mechanism is used to update the coordinates of the center point, and the noise points are not eliminated, but the influence of noise on subsequent semantic segmentation is solved. In combination with the downsampling method provided in the embodiment of the present invention, the robustness to noise and density sparsity is improved. The downsampling algorithm provided in the embodiment of the present invention is universal and can be used in combination with other point cloud processing methods to improve the point cloud processing effect.

[0093] The embodiment of the present invention also provides a method for enhancing the feature information of neighbor points by using a self-attention mechanism. Figure 4 FIG. 1 is a schematic diagram of a process for enhancing feature information of neighbor points provided by an embodiment of the present invention. Figure 4 As shown, the method includes:

[0094] Step 401: construct a neighbor matrix according to the original feature information of each second neighbor point.

[0095] The original feature information of each second neighbor point can be combined to generate a neighbor matrix. In one embodiment of the present invention, the neighbor matrix can also be constructed in the following manner: determine the relative coordinate information of the second neighbor point relative to the current center point; splice the relative coordinate information of the second neighbor point and the original feature information to obtain the spliced ​​feature information of the second neighbor point; combine the spliced ​​feature information of each second neighbor point to generate a neighbor matrix. The above method takes into account the coordinate information of the second neighbor point relative to the current center point, which can make the enhancement effect of the neighbor point and the current center point better.

[0096] Step 402: Input the neighbor matrix into the self-attention model to obtain the query matrix, key matrix and value matrix.

[0097] In the output of the attention model, the query matrix corresponds to the query matrix, the key matrix corresponds to the key matrix, and the value matrix corresponds to the value matrix.

[0098] Step 403: Generate an attention matrix based on the query matrix and the key matrix.

[0099] Step 404: Generate a self-attention score based on the attention matrix and the value matrix.

[0100] Step 405: Obtain enhanced feature information of the second neighbor point based on the self-attention score and the original feature information of the second neighbor point.

[0101] First, the coordinate information of each second neighbor point is normalized, and the coordinates of the second neighbor point are subtracted from the coordinates of the current center point to obtain the relative coordinate information of the second neighbor point; the feature information of the second neighbor point is recombined, that is, the relative coordinate information of the second neighbor point and the original feature information are spliced ​​to obtain the spliced ​​feature information; the spliced ​​feature information of each second neighbor point is combined to generate a neighbor matrix.

[0102] represents the set of multiple center points obtained using the inverse density value, and P s Contains noise points. Represents a collection of feature information of multiple center points. i , f i They represent the coordinate information and feature information of a central point respectively.

[0103] Use the KNN (K-Nearest Neighbor) algorithm to get multiple second neighbor points of the current center point and define the center point x i The coordinates of each second neighbor point are x i1 ,…,x ik ∈N(x i), the original feature information of each second neighbor point is f i1 ,…f ik .

[0104] The self-attention mechanism is used to solve the neighbor matrix to obtain the query matrix, key matrix and value matrix. The formula for using the self-attention mechanism to enhance the features of the second neighbor point is as follows:

[0105]

[0106] in:

[0107] γ(x i，j )=W γ f i，j

[0108]

[0109] In the above formula, the enhanced feature information of the second neighbor point is f′ i1 ,…f' ik φ(f i,k ) represents the query matrix, θ(f i,j ) represents the key matrix, W γ represents the value matrix, R(x i,k ,x i,j ) is the calculation score matrix. Perform dot multiplication of the query matrix and the key matrix, and then use the softmax function to generate the calculation score matrix. Multiply the attention matrix (attention matrix) and the value matrix corresponding to the calculation score matrix with the original feature of the second neighbor point to obtain the enhanced feature information of the second neighbor point. Among them, the self-attention score is obtained by multiplying the attention matrix and the value matrix; the self-attention score is multiplied by the original feature of the second neighbor point to obtain the enhanced feature information of the second neighbor point.

[0110] In one embodiment of the present invention, the coordinate information and original feature information of the current center point are updated based on the enhanced feature information of each second neighbor point, including: constructing an enhanced neighbor matrix based on the enhanced feature information of each second neighbor point; inputting the enhanced neighbor matrix into the first perception model to obtain a first output matrix; normalizing the first output matrix to generate a coordinate update matrix; and updating the coordinate information of the current center point based on the coordinate update matrix.

[0111] In one embodiment of the present invention, the coordinate information and original feature information of the current center point are updated based on the enhanced feature information of each second neighbor point, including: constructing an enhanced neighbor matrix based on the enhanced feature information of each second neighbor point; inputting the enhanced neighbor matrix into the second perception model to obtain a second output matrix; normalizing the second output matrix to generate a feature update matrix; and updating the feature information of the current center point based on the feature update matrix.

[0112] The idea of ​​the embodiment of the present invention is to use the enhanced feature information of the second neighbor point to learn and adjust the coordinate information and feature information of the center point. After the neighbor point features are enhanced through the self-attention mechanism, a new weight matrix is ​​obtained through the perception model. Then, the feature weight matrix and the coordinate weight matrix are calculated through the softmax function.

[0113] Assume f′ i,k is the feature information of the enhanced second neighbor point. The center point has K second neighbor points, so there is the following formula:

[0114]

[0115]

[0116] Among them, σ p ,σ f Represent the first perception model and the second perception model respectively. The first perception model and the second perception model can be constructed based on MLP (multi-layer perceptron). p 、F f Represent the first output matrix and the second output matrix respectively. p , W f Represent the coordinate update matrix and feature update matrix respectively. p , W f , get the updated center point coordinate information and feature information.

[0117] Using W p , update the coordinate information of the center point. p Add the coordinate information of the center point element by element to obtain the updated coordinate information. Specifically, for each channel, W p The value obtained under the channel is added to the value obtained by the center point coordinate information under the channel to obtain the updated coordinate information under the channel.

[0118] Using W f , update the feature information of the center point. f Add the original feature information of the center point element by element to obtain the updated feature information. Specifically, for each channel, W fThe value under the channel is added to the value of the center point feature information under the channel to obtain the updated feature information under the channel.

[0119] Figure 5 FIG. 1 is a schematic diagram of updating the coordinate information and feature information of a center point by using enhanced neighbor points provided by an embodiment of the present invention. Figure 5 As shown, Figure 5 There are 4 small pictures in the figure. The first small picture corresponds to the situation before the enhancement process is performed. The center point is far away from other neighboring points, and the center point is a noise point. In the second small picture, each neighboring point uses the self-attention mechanism to update the features. In the third small picture, the center point uses the enhanced feature information of the updated neighboring points for learning. In the fourth small picture, the center point updates the coordinate information and feature information. The center point is updated to a position closer to other neighboring points. The updated center point is close to the real data in the point cloud. By adjusting the coordinate information and feature information of the center point, the adverse effects of noise points are reduced and the accuracy of subsequent semantic recognition is improved.

[0120] In order to verify the effect of the downsampling method provided by the embodiment of the present invention, three scenes in the test set are selected for effect analysis. Figure 6a FIG. 1 is a schematic diagram of comparing the implementation effects of a downsampling algorithm provided by an embodiment of the present invention. Figure 6a As shown in the figure, Ground Truth represents real data, PointWeb is a downsampling method in the prior art, and PointWeb++ represents a downsampling method provided by an embodiment of the present invention. The real data comes from S3DIS (Stanford Large-Scale 3D Indoor Spaces Dataset). Figure 6a It can be seen that the method of the embodiment of the present invention achieves better downsampling effects than the PointWeb algorithm for both the window category and the column category by adding a center point update strategy.

[0121] Figure 6b FIG. 1 is a schematic diagram of another downsampling algorithm implementation effect comparison provided by an embodiment of the present invention. Figure 6bAs shown, the first row represents the first example diagram, the second row represents the second example diagram, and the third row represents the third example diagram. In the first example diagram, the method of the embodiment of the present invention can clearly separate the boundary between the window and the beam. In the second example diagram, the method of the embodiment of the present invention does not confuse the beam category with the ceiling category. In the third example diagram, after the center point update strategy is added to the method of the embodiment of the present invention, better sampling results are achieved for various categories such as board and clutter, which can improve the accuracy of subsequent semantic recognition, reflecting the superiority of the solution of the embodiment of the present invention.

[0122] The method of the embodiment of the present invention has a good sampling effect. Table 1 is a summary table of mIoU (mean intersection over union) values ​​of multiple downsampling methods. mIoU is an evaluation criterion for semantic segmentation. The larger the mIoU value, the better the model effect.

[0123] Table 1 Summary of mIoU values ​​of downsampling methods

[0124]

[0125] As shown in Table 1, PointNet, SegCloud, PointNet++, RSNet, etc. are all downsampling methods in the prior art. PointWeb++ represents the downsampling method provided by an embodiment of the present invention. The values ​​in Table 1 represent the mIoU values ​​of a certain downsampling method for a certain category. The comprehensive statistical value is the overall mIoU value of a certain downsampling method. For example, using the PointNet method, the mIoU value for the wall category is 69.8, the mIoU value for the beam category is 0.05, etc.

[0126] As can be seen from Table 1, compared with the PointWeb method, the overall mIoU value of the downsampling method provided by the embodiment of the present invention is improved by 1.24%, and there is a significant improvement in categories such as floor, table, and sofa. Among them, the table category achieves the best segmentation effect among all algorithms.

[0127] The robustness of the downsampling method provided by the embodiment of the present invention is also verified. Random data loss processing is performed on the ShapNet data set, and the loss rate is set to 0.5-1.0 and the step size is 0.1. Figure 7a This is a line chart of robustness verification of multiple algorithms provided by an embodiment of the present invention. Figure 7aAs shown in the figure, Robustness verification represents robustness verification, and drop ratio represents drop ratio. PointWeb2 represents the method of the embodiment of the present invention. PointWeb algorithm and DGCNN algorithm are two downsampling methods in the prior art. It can be seen that under different loss rates, the method provided by the embodiment of the present invention has a higher recognition accuracy.

[0128] In addition, noise is added to the object-level dataset, and 0, 1, 10, 50, and 100 noise points are added respectively for testing on the ModelNet dataset to verify the robustness of the method of the embodiment of the present invention to noise. Figure 7b This is another line chart of robustness verification of multiple algorithms provided by an embodiment of the present invention. Figure 7b As shown, Noise Points represents noise points. It can be seen that when the data set contains many noise points, the method of the embodiment of the present invention can still maintain good robustness, which fully verifies that the method for updating the feature information and coordinate information of the center point provided by the embodiment of the present invention has good robustness to noise.

[0129] Figure 8 FIG. 1 is a schematic diagram of a point cloud downsampling device provided by an embodiment of the present invention. Figure 8 As shown, the device comprises:

[0130] A point determination module 801 is used to determine a plurality of target points in a target point cloud;

[0131] The value determination module 802 is used to determine, for each target point, at least one first neighbor point of the target point from the target point cloud; and determine the inverse density value of the target point according to the distance between the target point and each first neighbor point;

[0132] A point selection module 803 is used to select a plurality of center points from a plurality of target points according to the inverse density value of each target point;

[0133] The down-sampling module 804 is used to perform down-sampling processing on the target point cloud according to multiple center points.

[0134] Optionally, the downsampling module 804 is specifically used for:

[0135] Determine the current center point from multiple center points;

[0136] Determine at least one second neighbor point of the current center point from the multiple center points;

[0137] Using the self-attention mechanism, the enhanced feature information of each second neighbor point is determined;

[0138] According to the enhanced feature information of each second neighbor point, the coordinate information and the original feature information of the current center point are updated;

[0139] The updated current center point is determined as the down-sampling point corresponding to the target point cloud, so as to perform down-sampling processing on the target point cloud.

[0140] Optionally, the downsampling module 804 is further configured to:

[0141] Construct a neighbor matrix according to the original feature information of each second neighbor point;

[0142] Input the neighbor matrix into the self-attention model to obtain the query matrix, key matrix and value matrix;

[0143] Generate an attention matrix based on the query matrix and the key matrix;

[0144] Generate self-attention scores based on the attention matrix and value matrix;

[0145] According to the self-attention score and the original feature information of the second neighbor point, the enhanced feature information of the second neighbor point is obtained.

[0146] Optionally, the downsampling module 804 is further configured to:

[0147] Determine the relative coordinate information of the second neighbor point relative to the current center point;

[0148] splicing the relative coordinate information of the second neighbor point and the original feature information to obtain splicing feature information of the second neighbor point;

[0149] The concatenated feature information of each second neighbor point is combined to generate a neighbor matrix.

[0150] Optionally, the downsampling module 804 is further configured to:

[0151] Constructing an enhanced neighbor matrix according to the enhanced feature information of each second neighbor point;

[0152] Inputting the enhanced neighbor matrix into the first perception model to obtain a first output matrix;

[0153] Normalizing the first output matrix to generate a coordinate update matrix;

[0154] According to the coordinate update matrix, the coordinate information of the current center point is updated.

[0155] Optionally, the downsampling module 804 is further configured to:

[0156] Constructing an enhanced neighbor matrix according to the enhanced feature information of each second neighbor point;

[0157] Inputting the enhanced neighbor matrix into the second perception model to obtain a second output matrix;

[0158] Normalizing the second output matrix to generate a feature update matrix;

[0159] According to the feature update matrix, the feature information of the current center point is updated.

[0160] Optionally, the downsampling module 804 is specifically used for:

[0161] The multiple center points are used as down-sampling points corresponding to the target point cloud, and the target point cloud is down-sampled.

[0162] An embodiment of the present invention provides an electronic device, including:

[0163] one or more processors;

[0164] a storage device for storing one or more programs,

[0165] When one or more programs are executed by one or more processors, the one or more processors implement the method of any of the above embodiments.

[0166] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system 900 of a terminal device suitable for implementing an embodiment of the present invention. Fig. 9 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0167] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0168] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0169] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are executed.

[0170] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0171] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0172] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, they may be described as: a point determination module, a value determination module, a point selection module, and a downsampling module. The names of these modules do not, in some cases, constitute limitations on the modules themselves, for example, the point determination module may also be described as a "module for determining multiple target points in a target point cloud".

[0173] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes:

[0174] For each of the target points, determining at least one first neighbor point of the target point from the target point cloud; determining an inverse density value of the target point according to the distance between the target point and each of the first neighbor points;

[0175] Selecting a plurality of center points from the plurality of target points according to the inverse density value of each of the target points;

[0176] The target point cloud is downsampled according to the multiple center points.

[0177] According to the technical solution of the embodiment of the present invention, the inverse density value of the target point is determined according to the distance between the target point and each of the first neighboring points of the target point. The inverse density value is a statistical value of the distance between the target point and each of the first neighboring points, such as the sum, average value, mode, etc. The inverse density value can reflect the distribution of each target point in the target point cloud. Compared with the random sampling method, the solution of the embodiment of the present invention selects multiple center points through the inverse density value to perform down-sampling processing on the target point cloud, which can select the down-sampling points more evenly, retain important information in the point cloud, and reduce the loss of key point data after down-sampling.

[0178] In addition, determining the first neighbor points of the target point and calculating the distance between the target point and the first neighbor points are usually necessary steps for processing point clouds. Therefore, the solution of the embodiment of the present invention can be directly implemented using the calculated distances between the target point and the first neighbor points, which has a high execution efficiency.

[0179] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A point cloud downsampling method, characterized in that: include: Determine multiple target points in the target point cloud; For each of the target points, determining at least one first neighbor point of the target point from the target point cloud; determining an inverse density value of the target point according to the distance between the target point and each of the first neighbor points; Selecting a plurality of center points from the plurality of target points according to the inverse density value of each of the target points; The target point cloud is downsampled according to the multiple center points.

2. The method according to claim 1, characterized in that The down-sampling process of the target point cloud according to the multiple center points includes: Determine a current center point from the multiple center points; Determine at least one second neighbor point of the current center point from the multiple center points; Determine enhanced feature information of each of the second neighbor points by using a self-attention mechanism; updating the coordinate information and original feature information of the current center point according to the enhanced feature information of each of the second neighboring points; The updated current center point is determined as a down-sampling point corresponding to the target point cloud, so as to perform down-sampling processing on the target point cloud.

3. The method according to claim 2, characterized in that: The using the self-attention mechanism to determine the enhanced feature information of each of the second neighbor points includes: Constructing a neighbor matrix according to the original feature information of each of the second neighbor points; Input the neighbor matrix into the self-attention model to obtain a query matrix, a key matrix and a value matrix; Generate an attention matrix according to the query matrix and the key matrix; Generate a self-attention score according to the attention matrix and the value matrix; According to the self-attention score and the original feature information of the second neighbor point, enhanced feature information of the second neighbor point is obtained.

4. The method according to claim 3, characterized in that: The step of constructing a neighbor matrix according to the original feature information of each of the second neighbor points includes: Determine relative coordinate information of the second neighbor point relative to the current center point; splicing the relative coordinate information and the original feature information of the second neighbor point to obtain splicing feature information of the second neighbor point; The splicing feature information of each of the second neighbor points is combined to generate the neighbor matrix.

5. The method according to claim 2, characterized in that: The updating process of the coordinate information and the original feature information of the current center point according to the enhanced feature information of each of the second neighboring points includes: Constructing an enhanced neighbor matrix according to the enhanced feature information of each of the second neighbor points; Inputting the enhanced neighbor matrix into a first perception model to obtain a first output matrix; Normalizing the first output matrix to generate a coordinate update matrix; The coordinate information of the current center point is updated according to the coordinate update matrix.

6. The method according to claim 2, characterized in that: The updating process of the coordinate information and the original feature information of the current center point according to the enhanced feature information of each of the second neighboring points includes: Constructing an enhanced neighbor matrix according to the enhanced feature information of each of the second neighbor points; Inputting the enhanced neighbor matrix into a second perception model to obtain a second output matrix; Normalizing the second output matrix to generate a feature update matrix; The feature information of the current center point is updated according to the feature update matrix.

7. The method according to claim 1, characterized in that The down-sampling process of the target point cloud according to the multiple center points includes: The multiple center points are used as down-sampling points corresponding to the target point cloud, and down-sampling processing is performed on the target point cloud.

8. A point cloud downsampling device, characterized in that: include: A point determination module, used to determine multiple target points in the target point cloud; a value determination module, configured to determine, for each of the target points, at least one first neighbor point of the target point from the target point cloud; and determine an inverse density value of the target point according to a distance between the target point and each of the first neighbor points; A point selection module, configured to select a plurality of center points from the plurality of target points according to the inverse density value of each of the target points; A downsampling module is used to downsample the target point cloud according to the multiple center points.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images