Point cloud data gridding method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides a point cloud data gridding method and device, electronic equipment and a storage medium. The method comprises the steps of performing feature extraction on to-be-predicted point cloud data based on a pre-trained encoder to obtain intrinsic information of the to-be-predicted point cloud data; inputting the intrinsic information of the to-be-predicted point cloud data into a pre-trained edge prediction model to obtain edge prediction information of the to-be-predicted point cloud data; and generating gridding data corresponding to the to-be-predicted point cloud data based on the edge prediction information of the to-be-predicted point cloud data. According to the point cloud data gridding method and device, the electronic equipment and the storage medium, dependence of a traditional method on manual intervention can be avoided, the calculation complexity is low, and the method and device are suitable for processing point cloud data of various structures.

Description

Technical Field

[0001] This disclosure pertains to the field of artificial intelligence, and particularly relates to a method, apparatus, electronic device, storage medium, and computer program product for point cloud data meshing. Background Technology

[0002] With the development of artificial intelligence technology, point cloud data has been widely used in 3D modeling, autonomous driving, robot navigation, digital twins, virtual reality, and other fields. The conversion of point cloud data to meshed data, as a key step in data processing and modeling, can transform discrete point cloud data into a continuous surface representation with geometric and topological relationships, thereby better supporting visualization analysis and interaction. Existing technologies commonly used for converting point cloud data to meshed data mainly include sphere rotation methods and Poisson surface reconstruction methods. Summary of the Invention

[0003] This disclosure provides a point cloud data meshing method, apparatus, electronic device, storage medium, and computer program product, which can avoid the dependence on manual intervention in traditional methods, have low computational complexity, and are suitable for processing point cloud data of various structures.

[0004] In a first aspect, embodiments of this disclosure provide a point cloud data gridding method, the method comprising: extracting features from point cloud data to be predicted based on a pre-trained encoder to obtain intrinsic information of the point cloud data to be predicted; inputting the intrinsic information of the point cloud data to be predicted into a pre-trained edge prediction model to obtain edge prediction information of the point cloud data to be predicted; and generating gridded data corresponding to the point cloud data to be predicted based on the edge prediction information of the point cloud data to be predicted.

[0005] In one feasible implementation, the pre-trained edge prediction model includes a state update unit and an edge prediction unit, which are used to predict the edge prediction information of the point cloud data to be predicted sequentially based on the intrinsic information of the point cloud data to be predicted, according to the state update unit and the edge prediction unit; the edge prediction unit is used to predict the edge prediction information of the current point based on the current interconnection state; the state update unit is used to update the current interconnection state based on the edge prediction information of the current point.

[0006] In one feasible implementation, the intrinsic information of the point cloud data to be predicted includes the intrinsic information of each point in the point cloud data to be predicted; the edge prediction information of the point cloud data to be predicted includes the edge prediction information of each point in the point cloud data to be predicted; inputting the intrinsic information of the point cloud data to be predicted into a pre-trained edge prediction model to obtain the edge prediction information of the point cloud data to be predicted includes: inputting the intrinsic information of each point in the point cloud data to be predicted into the pre-trained edge prediction model in a first preset order to obtain the edge prediction information of each point in the point cloud data to be predicted.

[0007] In one feasible implementation, the first preset order is obtained by sorting each point in the point cloud data to be predicted using a traversal algorithm; feature extraction is performed on the point cloud data to be predicted based on a pre-trained encoder to obtain the intrinsic information of the point cloud data to be predicted, including: feature extraction is performed on each point in the point cloud data to be predicted according to the first preset order based on the pre-trained encoder to obtain the intrinsic information of each point in the point cloud data to be predicted.

[0008] In one feasible implementation, the state update unit includes a first long short-term memory network; the edge prediction unit includes a second long short-term memory network; according to a first preset order, the intrinsic information of each point in the point cloud data to be predicted is input into a pre-trained edge prediction model to obtain the edge prediction information of each point in the point cloud data to be predicted, including: repeatedly executing the following steps until the edge prediction information of each point in the point cloud data to be predicted is obtained: determining the current point from the point cloud data to be predicted according to the first preset order; inputting the intrinsic information of the current point into the second long short-term memory network according to the current interconnection state in the first long short-term memory network to obtain the edge prediction information of the current point; updating the current interconnection state stored in the first long short-term memory network according to the current point and the edge prediction information of the current point.

[0009] In one feasible implementation, the intrinsic information of the current point includes spatial feature information; the second long short-term memory network is used to evaluate the spatial correlation between the current point and the current interconnection state based on the spatial feature information of the current point, and to determine the edge prediction information of the current point based on the spatial correlation.

[0010] In one feasible implementation, before extracting features from the point cloud data to be predicted based on a pre-trained encoder to obtain the intrinsic information of the point cloud data to be predicted, the method further includes: filtering and denoising the original point cloud data to obtain effective point cloud data; and standardizing the coordinate information of the effective point cloud data to obtain the point cloud data to be predicted.

[0011] In one feasible implementation, the pre-trained encoder is obtained by training the autoencoder based on the first training point cloud data; the autoencoder includes an encoder to be trained and a decoder; the encoder to be trained is used to extract features from the first training point cloud data to obtain first training intrinsic information; the decoder is used to reconstruct the first training point cloud data based on the first training intrinsic information output by the encoder to be trained, with the training objective of minimizing the difference between the first training point cloud data and the reconstructed first training point cloud data.

[0012] In one feasible implementation, the method further includes: optimizing the gridded data to obtain optimized gridded data.

[0013] In one feasible implementation, the gridded data is smoothed to obtain intermediate gridded data; the intermediate gridded data is then enhanced with detail to obtain optimized gridded data.

[0014] Secondly, embodiments of this disclosure provide a point cloud data meshing device, the device comprising: The feature extraction module is used to extract features from the point cloud data to be predicted based on the pre-trained encoder, so as to obtain the intrinsic information of the point cloud data to be predicted. The prediction module is used to input the intrinsic information of the point cloud data to be predicted into the pre-trained edge prediction model to obtain the edge prediction information of the point cloud data to be predicted. The generation module is used to generate gridded data corresponding to the point cloud data to be predicted based on the edge prediction information of the point cloud data to be predicted.

[0015] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the point cloud data meshing method of any of the embodiments of the first aspect described above.

[0016] Fourthly, embodiments of this disclosure provide a computer storage medium storing computer program instructions, which, when executed by a processor, implement the point cloud data meshing method of any of the embodiments in the first aspect described above.

[0017] Fifthly, embodiments of the present invention provide a computer program product, including a computer program that, when executed, implements the point cloud data meshing method of any of the embodiments of the first aspect described above.

[0018] The point cloud data meshing method, apparatus, electronic device, storage medium, and computer program product of this disclosure first extract features from the point cloud data to be predicted based on a pre-trained encoder to obtain the intrinsic information of the point cloud data to be predicted. This intrinsic information deeply integrates the geometric and topological features of the point cloud data, which can provide a reliable feature basis for subsequent edge prediction. Subsequently, the intrinsic information of the point cloud data to be predicted is input into a pre-trained edge prediction model. Based on the state update unit and edge prediction unit in the pre-trained edge prediction model, the edge prediction information of the point cloud data to be predicted is predicted sequentially. In this way, based on the adaptive learning capability of the deep learning model, it can accurately capture the complex geometric structure and topological relationship in the point cloud data, and has good robustness to noise, non-uniform density, and scenes with sharp features. Based on the edge prediction information of the point cloud data to be predicted, meshed data corresponding to the point cloud data to be predicted is generated. In this way, through an end-to-end automated process, meshed data can be directly generated from the original point cloud data, avoiding the dependence on manual intervention of traditional methods, with low computational complexity, and suitable for processing point cloud data with various structures. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments of the present invention will be briefly introduced below. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating a point cloud data meshing method provided in an embodiment of this disclosure; Figure 2 This is a flowchart illustrating a point cloud data meshing method provided in an embodiment of this disclosure; Figure 3 This is a flowchart illustrating a point cloud data meshing method provided in an embodiment of this disclosure; Figure 4 This is a flowchart illustrating a point cloud data meshing method provided in an embodiment of this disclosure; Figure 5 This is a schematic diagram of the structure of a point cloud data meshing device provided in an embodiment of this disclosure; Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] The features and exemplary embodiments of various aspects of the present invention will now be described in detail. To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely intended to explain the present invention and not to limit the present invention. For those skilled in the art, the present invention can be practiced without some of these specific details. The following description of the embodiments is merely to provide a better understanding of the present invention by illustrating examples of the invention.

[0022] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0023] In existing technologies, traditional point cloud data to gridded data conversion techniques mainly include sphere rotation methods and Poisson surface reconstruction methods. However, these traditional methods suffer from problems such as high computational complexity, difficulty in handling large-scale data, low fidelity, and sensitivity to noise and incomplete data.

[0024] In the sphere rotation method, the generated mesh is typically constructed based on a spherical neighborhood, making it sensitive to the selection of seed points and significantly impacting performance. Uneven distribution or noise in the dataset can lead to inaccurate sphere trajectories, affecting algorithm performance. For example, when point data for a given point is sparse, the sphere rotation algorithm cannot connect distant points, resulting in a mesh with gaps or holes.

[0025] Poisson surface reconstruction methods require solving the Poisson equation, which involves complex numerical calculations and large-scale matrix solving. Therefore, the computational complexity is high when processing large-scale point cloud data, and it may require a long computation time. Poisson surface reconstruction methods also have high requirements for the estimation of the normal vectors in the point cloud data. If the normal vector estimation is inaccurate or contains noise, it may lead to distortion or incomplete reconstruction results. Furthermore, this method is suitable for smooth closed surfaces, but its performance is not ideal for point cloud data with sharp features or non-smooth surfaces, requiring additional optimization processing.

[0026] To address the problems of the prior art, this disclosure provides a point cloud data meshing method, apparatus, electronic device, storage medium, and computer program product.

[0027] The point cloud data gridding method provided in the embodiments of this disclosure will be introduced first.

[0028] It should be noted that the mesh consists of vertices, edges, and faces. By predicting the interconnecting edges between each point and the rest of the points, a network structure containing vertices, edges, and faces can be obtained to generate meshed data.

[0029] Figure 1 This is a flowchart illustrating a point cloud data meshing method provided in an embodiment of this disclosure. Figure 1 As shown, the method may include the following steps: Step S101: Based on the pre-trained encoder, feature extraction is performed on the point cloud data to be predicted to obtain the intrinsic information of the point cloud data to be predicted. Step S102: Input the intrinsic information of the point cloud data to be predicted into the pre-trained edge prediction model to obtain the edge prediction information of the point cloud data to be predicted. Step S103: Generate gridded data corresponding to the point cloud data to be predicted based on the edge prediction information of the point cloud data to be predicted.

[0030] In some embodiments, the original point cloud data is filtered and denoised to obtain effective point cloud data; the coordinate information of the effective point cloud data is standardized to obtain the point cloud data to be predicted.

[0031] For example, Gaussian filtering is applied to the original point cloud data, and noise points that deviate from the normal range are removed based on the coordinate distance between adjacent points to obtain effective point cloud data. The coordinate information of each point in the effective point cloud data is standardized to obtain the point cloud data to be predicted, so as to eliminate the difference in coordinate scale under different acquisition scenarios. The standardization process can be to translate the coordinate information of all points to the origin or scale them to a uniform coordinate range.

[0032] In some embodiments, before extracting features from the point cloud data to be predicted using a pre-trained encoder to obtain the intrinsic information of the point cloud data, a PointNet++ network is used to divide the point cloud data to be predicted into multiple local regions based on the coordinate information of each point in the point cloud data. Statistical features of the coordinate information of each local region are calculated and used as local feature information. Feature extraction is then performed on the coordinate information and local feature information of each point in the point cloud data to be predicted based on the pre-trained encoder to obtain the intrinsic information of each point in the point cloud data to be predicted. For example, the point cloud data to be predicted is divided into multiple local regions based on the K-nearest neighbor algorithm; the mean of the coordinate information of each local region is calculated and used as the local feature information of each point in that local region; feature extraction is then performed on the coordinate information and local feature information of each point in the point cloud data to be predicted based on the pre-trained encoder to obtain the intrinsic information of each point in the point cloud data to be predicted. This way, even if the point cloud data has slight sparsity or uneven density, local information can still be supplemented based on the statistical features of the coordinates, reducing the distortion of intrinsic information caused by incomplete data and improving the robustness of feature extraction.

[0033] It should be noted that the intrinsic information of the point cloud data to be predicted is used to characterize the key features and structural information of the point cloud data. It includes the global spatial distribution and local details of the point cloud data. Learning the intrinsic information can obtain an efficient feature representation of the point cloud data. For example, the intrinsic information of each point in the point cloud data to be predicted can be an intrinsic tensor.

[0034] It should be noted that in some embodiments, the encoder can map high-dimensional raw data to a low-dimensional feature space. The decoder remaps the low-dimensional features output by the encoder back to the original data to verify the validity of the low-dimensional features extracted by the encoder.

[0035] In some embodiments, the pre-trained encoder is obtained by training the autoencoder based on first training point cloud data. Exemplarily, the autoencoder includes convolutional layers, batch normalization layers, and fully connected layers.

[0036] In some embodiments, the autoencoder includes an encoder to be trained and a decoder; the encoder to be trained is used to extract features from the first training point cloud data to obtain first training intrinsic information; the decoder is used to reconstruct the first training point cloud data based on the first training intrinsic information output by the encoder to be trained, with the training objective of minimizing the difference between the first training point cloud data and the reconstructed first training point cloud data.

[0037] In some embodiments, the pre-trained edge prediction model includes a state update unit and an edge prediction unit; the edge prediction unit is used to predict the edge prediction information of the current point based on the current interconnection state; the state update unit is used to update the current interconnection state based on the edge prediction information of the current point.

[0038] It should be noted that Long Short Term Memory (LSTM) is a deep learning model used to process sequential data. It can effectively convey and express information in long sequence data without forgetting useful information from earlier periods.

[0039] In some embodiments, both the state update unit and the edge prediction unit may include a long short-term memory network.

[0040] It should be noted that in some embodiments, after all points in the point cloud data to be predicted have been introduced and the edges between them have also been predicted, a network structure composed of points and edges is obtained. For example, the gridded data consists of triangular faces; when three points are connected by three predicted edges, these three points and their interconnecting edges generate a triangular face.

[0041] In some embodiments of this disclosure, feature extraction is first performed on the point cloud data to be predicted based on a pre-trained encoder to obtain intrinsic information of the point cloud data to be predicted. This intrinsic information deeply integrates the geometric and topological features of the point cloud data, which can provide a reliable feature basis for subsequent edge prediction. Subsequently, the intrinsic information of the point cloud data to be predicted is input into a pre-trained edge prediction model. Based on the state update unit and edge prediction unit in the pre-trained edge prediction model, the edge prediction information of the point cloud data to be predicted is predicted sequentially. In this way, based on the adaptive learning capability of the deep learning model, the complex geometric structure and topological relationship in the point cloud data can be accurately captured, and it has good robustness to noise, non-uniform density and scenes with sharp features. Based on the edge prediction information of the point cloud data to be predicted, gridded data corresponding to the point cloud data to be predicted is generated. In this way, through an end-to-end automated process, gridded data can be directly generated from the original point cloud data, avoiding the dependence on manual intervention of traditional methods, with low computational complexity, and suitable for processing point cloud data with various structures.

[0042] Figure 2 This is a flowchart illustrating a point cloud data meshing method provided in an embodiment of this disclosure. Figure 2 As shown, the method may include the following steps: Step S201: Based on the pre-trained encoder, feature extraction is performed on each point in the point cloud data to be predicted in a first preset order to obtain the intrinsic information of each point in the point cloud data to be predicted. Step S202: According to the first preset order, the intrinsic information of each point in the point cloud data to be predicted is input into the pre-trained edge prediction model to obtain the edge prediction information of each point in the point cloud data to be predicted.

[0043] Step S103: Generate gridded data corresponding to the point cloud data to be predicted based on the edge prediction information of the point cloud data to be predicted.

[0044] It should be noted that, in some embodiments, the first preset order can be an ordered point sequence, used to represent the order in which each point in the point cloud data to be predicted is introduced; by using the first preset order, the pre-trained edge prediction model can process points in sequence, capture the sequential dependencies between points, and provide a logical order for the temporal input of the edge prediction model, avoiding the inability of the model to learn the spatial structure of the point cloud data due to the disordered input of points.

[0045] In some embodiments, the first preset order is obtained by sorting each point in the point cloud data to be predicted using a traversal algorithm. For example, a point cloud coordinate matrix is ​​constructed to store the coordinate information of each point in the point cloud data to be predicted; the point cloud coordinate matrix is ​​traversed using a traversal algorithm to obtain the first preset order.

[0046] In some embodiments, a depth-first search method is used to traverse each point in the point cloud data to be predicted and determine a first preset order. For example, during the traversal, the coordinate information and sequence number of each point are recorded sequentially; traversal points are randomly selected from the point cloud data to be predicted; the coordinate distances between the traversal point and all other points are calculated, and based on these distances, a set of neighboring points for the traversal point is determined; an untraversed point is selected from the neighboring point set as a new traversal point, and the calculation of the coordinate distances between this traversal point and all other points is returned; if no untraversed point exists in the neighboring point set of a traversal point, the traversal proceeds back to the previous node.

[0047] It should be noted that in some embodiments, using a depth-first search method for traversal can ensure that adjacent points are spatially close, making it easier to identify real spatial connections in subsequent edge prediction, reducing erroneous connections across regions, and improving the structural rationality of gridded data.

[0048] In some embodiments of this disclosure, based on a pre-trained encoder, features are extracted from each point in the point cloud data to be predicted in a first preset order to obtain intrinsic information of each point in the point cloud data to be predicted; the intrinsic information of each point in the point cloud data to be predicted is input into a pre-trained edge prediction model in the first preset order to obtain edge prediction information of each point in the point cloud data to be predicted; gridded data corresponding to the point cloud data to be predicted is generated based on the edge prediction information of the point cloud data to be predicted. In this way, the generated ordered point sequence can be used to provide a logical order for the temporal input of the edge prediction model, avoiding the inability of the edge prediction model to learn the spatial structure of the point cloud data due to the disordered input of points, ensuring the spatial coherence of the gridded data, and improving the structural rationality of the gridded data.

[0049] It should be noted that, in some embodiments, the state update unit includes a first long short-term memory network; the edge prediction unit includes a second long short-term memory network.

[0050] Figure 3 This is a flowchart illustrating a point cloud data meshing method provided in an embodiment of this disclosure. Figure 3 As shown, the method may include the following steps: Step S201: Based on the pre-trained encoder, feature extraction is performed on each point in the point cloud data to be predicted in a first preset order to obtain the intrinsic information of each point in the point cloud data to be predicted. Step S301: Determine the current point from the point cloud data to be predicted according to the first preset order; Step S302: Based on the current interconnection state in the first long short-term memory network, input the intrinsic information of the current point into the second long short-term memory network to obtain the edge prediction information of the current point; Step S303: Update the current interconnection state stored in the first long short-term memory network based on the current point and the edge prediction information of the current point; Step S304: Determine whether edge prediction information for each point in the point cloud data to be predicted has been obtained; if yes, proceed to step S103; otherwise, proceed to step S301.

[0051] Step S203: Generate gridded data corresponding to the point cloud data to be predicted based on the edge prediction information of the point cloud data to be predicted.

[0052] It should be noted that in some embodiments, the first preset order is obtained by sorting each point in the point cloud data to be predicted using a traversal algorithm.

[0053] For example, updating the current interconnection state stored in the first long short-term memory network based on the current point and the edge prediction information of the current point can be expressed by the following formula (1): (1) in, For the first The current interconnection state, i.e., the updated current interconnection state; For the first Current interconnection status; For the first Edge prediction information for each point; Let be the intrinsic tensor of the point cloud data to be predicted.

[0054] It should be noted that, in some embodiments, the intrinsic information of each point in the point cloud data to be predicted includes spatial feature information; the second long short-term memory network can be used to evaluate the spatial correlation between the current point and the current interconnection state based on the spatial feature information of the current point, and determine the edge prediction information of the current point based on the spatial correlation. For example, the second long short-term memory network can learn, through training, the spatial correlation between the current point and each point in the current interconnection state based on the spatial feature information of each point; if the spatial correlation between the current point and each point in the current interconnection state is greater than a preset threshold, it is determined that there is no connection between the two points; otherwise, it is determined that there is an edge between the two points; the preset threshold can be determined based on the physical size of the object.

[0055] In some embodiments, the second long short-term memory network can also learn, through training, whether the existence of an edge between the current point and the first predicted point conforms to the continuity of the object surface based on the current interconnection edge information of the first predicted point, in order to avoid generating predicted edges that violate spatial logic.

[0056] In some embodiments of this disclosure, the current point is sequentially determined from the point cloud data to be predicted. Based on the current interconnection state in the first long short-term memory network, the intrinsic information of the current point is input into the second long short-term memory network to obtain the edge prediction information of the current point. The current interconnection state stored in the first long short-term memory network is updated based on the current point and the edge prediction information of the current point. In this way, the edge prediction information can be dynamically accumulated in the current interconnection state and used as the context for predicting subsequent edges. This allows spatial constraints to be determined based on the coordinate information of each point in the point cloud data to be predicted, filtering out unreasonable connections in space, reducing redundant or dangling edges in the generated gridded data, so as to reduce the error edge prediction rate and improve edge fidelity, avoiding the smoothing distortion of sharp features in traditional methods.

[0057] Figure 4 This is a flowchart illustrating a point cloud data meshing method provided in an embodiment of this disclosure. Figure 4 As shown, the method may include the following steps: Step S101: Based on the pre-trained encoder, feature extraction is performed on the point cloud data to be predicted to obtain the intrinsic information of the point cloud data to be predicted. Step S102: Input the intrinsic information of the point cloud data to be predicted into the pre-trained edge prediction model to obtain the edge prediction information of the point cloud data to be predicted.

[0058] Step S103: Generate gridded data corresponding to the point cloud data to be predicted based on the edge prediction information of the point cloud data to be predicted; Step S401: Optimize the gridded data to obtain optimized gridded data.

[0059] It should be noted that, in some embodiments, smoothing the mesh data can reduce jagged distortion on the mesh surface caused by noise; and enhancing the details of the mesh data can preserve the key features of the object and improve the fidelity and usability of the mesh data.

[0060] In some embodiments, the gridded data is smoothed to obtain intermediate gridded data; the intermediate gridded data is then enhanced with detail to obtain optimized gridded data.

[0061] It should be noted that, in some embodiments, smoothing the meshed data can reduce surface undulations or jagged defects caused by point cloud noise and edge prediction errors by adjusting vertex coordinates or local surface morphology without destroying the overall structure of the mesh, so as to make the mesh surface more continuous and smooth, and more in line with the surface morphology of real objects.

[0062] Optionally, in some embodiments, vertex adjustment methods are used to smooth the meshed data to obtain intermediate meshed data.

[0063] In some embodiments, the vertex adjustment class method can be Laplacian smoothing; for each vertex in the gridded data, the mean of the coordinate information of all neighboring vertices of that vertex is calculated, and the coordinate information of that vertex is updated to the mean.

[0064] In some embodiments, the vertex adjustment method can be bilateral filtering smoothing; for each vertex in the gridded data, determine multiple neighboring vertices of that vertex; calculate the spatial weight and normal vector weight of each neighboring vertex; determine the total weight of each neighboring vertex based on the spatial weight and normal vector weight of each neighboring vertex; and update the coordinate information of the vertex based on the total weight of each neighboring vertex among the multiple neighboring vertices.

[0065] For example, the spatial weights of adjacent vertices can be calculated using the following formula (2): (2) in, Spatial weights of adjacent vertices; For the current vertex With adjacent vertices The distance; This refers to the spatial scale parameter.

[0066] For example, the normal vector weights of adjacent vertices can be calculated using the following formula (3): (3) in, The normal vector weights of adjacent vertices; For the current vertex With adjacent vertices The included angle; This refers to the spatial scale parameter.

[0067] For example, the total weight of adjacent vertices can be calculated using the following formula (4): (4) in, This represents the total weight of adjacent vertices.

[0068] For example, updating the coordinate information of a vertex based on the total weight of each of its multiple neighboring vertices can be represented by the following formula (5): (5) in, This is the updated coordinate information for that vertex; This refers to the coordinate information of adjacent vertices; As vertex The set of adjacent vertices.

[0069] Optionally, in some embodiments, a surface morphology optimization method is used to smooth the meshed data to obtain intermediate meshed data.

[0070] In some embodiments, a triangle remeshing method is used to split faces with excessively large aspect ratios or merge faces with excessively small areas to obtain intermediate meshed data. For example, triangles with side lengths exceeding a preset side length threshold are split into four triangles; while ensuring that the normal vectors are aligned after merging, triangles with areas smaller than a preset area threshold are merged into one triangle.

[0071] It should be noted that, in some embodiments, detail enhancement processing can be a process that restores and strengthens the fine structures lost during edge prediction based on the coordinate information and intrinsic information of the point cloud data to be predicted. Performing detail enhancement processing on intermediate gridded data can ensure the smoothness of the gridded data and accurately restore the original details of the point cloud data.

[0072] It should be noted that in some embodiments, the greater the local curvature of a vertex, the higher the degree of curvature of the region where the vertex is located, and there may be small depressions or sharp edges.

[0073] Optionally, in some embodiments, the intrinsic information of the point cloud data to be predicted includes local curvature information; based on the local curvature information, the intermediate gridded data is subjected to detail enhancement processing to obtain optimized gridded data.

[0074] For example, local curvature information of each vertex is extracted from the intrinsic tensor output by the PointNet++ network; for regions where the local curvature information is greater than a preset local curvature threshold, if the region is determined to be concave, the vertex coordinates of the center of the concavity in the region are finely adjusted inward to increase the depth of the concavity; if the region is determined to be convex, the vertex coordinates of the center of the convexity in the region are finely adjusted outward to enhance the sharpness of the edges; the offset of the fine adjustment does not exceed 0.5 times the average density of the point cloud data to be predicted, so as to avoid over-adjustment leading to distortion of details.

[0075] Optionally, in some embodiments, edge detection technology is used to enhance the details of the intermediate meshed data to obtain optimized meshed data. For example, for each vertex in the meshed data, the coordinate gradient information of each of its multiple neighboring vertices is calculated; neighboring vertices whose coordinate gradient information exceeds a preset coordinate gradient threshold are marked as edge vertices; the region where the edge vertex is located is enhanced with details using an edge length adjustment method or a normal vector correction method; in the edge length adjustment method, the length of the edges between edge vertices is shortened to make the triangles in the edge region denser, highlighting the sharpness of the edge; in the normal vector correction method, the angle between the normal vectors of the triangles on both sides of the edge is increased by adjusting the normal vectors on both sides to enhance the visual contrast of the edge.

[0076] Optionally, in some embodiments, the intermediate meshed data is enhanced with detail based on the coordinate information of the point cloud data to be predicted, resulting in optimized meshed data. For example, the number of vertices in the same region of the meshed data is compared with the number of points in the same region of the point cloud data to be predicted; if the number of vertices in the meshed data corresponding to that region is greater than the number of points in the point cloud data to be predicted, that region is determined to be a region of lost detail; for the region of lost detail, new triangles are generated by adding vertices and edges based on the coordinate information of the point cloud data to be predicted.

[0077] For example, the specific implementation of generating a new triangle face by adding vertices and edges based on the coordinate information of the point cloud data to be predicted can be as follows: select points in the point cloud data to be predicted that are not included in the gridded data as new vertices; determine the vertices connected to the new vertices based on the neighborhood relationship of the depth-first search technique, and generate the triangle face.

[0078] In some embodiments of this disclosure, feature extraction is performed on the point cloud data to be predicted based on a pre-trained encoder to obtain the intrinsic information of the point cloud data to be predicted; the intrinsic information of the point cloud data to be predicted is input into a pre-trained edge prediction model to obtain the edge prediction information of the point cloud data to be predicted; gridded data corresponding to the point cloud data to be predicted is generated based on the edge prediction information of the point cloud data to be predicted; the gridded data is optimized to obtain optimized gridded data. This can reduce the jagged distortion of the gridded data, retain the key features of the object, and improve the fidelity and usability of the gridded data.

[0079] It should be noted that the application scenarios of the point cloud data meshing method provided in this disclosure can include 3D modeling of indoor environments for service robots and 3D reconstruction of obstacles for autonomous driving.

[0080] In the application scenario of 3D modeling of indoor environments for service robots, service robots need to achieve autonomous navigation, obstacle avoidance and task execution through 3D mesh maps; the indoor environment includes objects such as walls, furniture and home appliances; its point cloud data has the following characteristics: the point cloud of planar areas such as walls and floors is dense but contains sensor noise, the point cloud of key detail areas such as table corners, door frames and edges of home appliances is sparse, and dynamic objects cause local point cloud interference.

[0081] For example, based on a pre-trained encoder, features are extracted from each point in the 3D map point cloud data of the service robot according to a first preset order to obtain the intrinsic information of each point in the 3D map point cloud data; the intrinsic information of each point in the 3D map point cloud data is input into a pre-trained edge prediction model according to the first preset order to obtain the edge prediction information of each point in the 3D map point cloud data; gridded data corresponding to the 3D map point cloud data is generated based on the edge prediction information of the 3D map point cloud data; the gridded data is optimized to obtain optimized gridded data. This enables the service robot to accurately identify table corners or door frames, achieve autonomous obstacle avoidance in narrow passages, avoid the edge blurring problem in traditional point cloud data gridding methods, and reduce the navigation collision rate.

[0082] In the application scenario of 3D obstacle reconstruction for autonomous driving, autonomous vehicles need to reconstruct obstacles such as pedestrians and vehicles in real time using LiDAR point cloud data for collision warning and path planning. In this application scenario, the point cloud of dynamic obstacle areas is sparse, details of obstacle edge areas are easily lost, and real-time response is required.

[0083] For example, based on a pre-trained encoder, features are extracted from each point in the autonomous driving obstacle point cloud data according to a first preset order to obtain the intrinsic information of each point in the autonomous driving obstacle point cloud data; according to the first preset order, the intrinsic information of each point in the autonomous driving obstacle point cloud data is input into a pre-trained edge prediction model to obtain the edge prediction information of each point in the autonomous driving obstacle point cloud data; based on the edge prediction information of the autonomous driving obstacle point cloud data, gridded data corresponding to the autonomous driving obstacle point cloud data is generated; the gridded data is optimized to obtain optimized gridded data, which can restore details such as pedestrian arms and vehicle door openings, thereby improving the accuracy of the collision warning system and supporting autonomous vehicles to safely avoid obstacles in complex road conditions.

[0084] Figure 5 This is a schematic diagram of the structure of a point cloud data meshing device provided in an embodiment of this disclosure. Figure 5 As shown, the device may include a feature extraction module 510, a prediction module 520, and a generation module 530.

[0085] Among them, the feature extraction module 510 is used to extract features from the point cloud data to be predicted based on the pre-trained encoder, so as to obtain the intrinsic information of the point cloud data to be predicted. The prediction module 520 is used to input the intrinsic information of the point cloud data to be predicted into the pre-trained side prediction model to obtain the side prediction information of the point cloud data to be predicted. The generation module 530 is used to generate gridded data corresponding to the point cloud data to be predicted based on the edge prediction information of the point cloud data to be predicted.

[0086] In some embodiments, the pre-trained edge prediction model includes a state update unit and an edge prediction unit, which are used to predict the edge prediction information of the point cloud data to be predicted sequentially based on the intrinsic information of the point cloud data to be predicted and the state update unit and the edge prediction unit; the edge prediction unit is used to predict the edge prediction information of the current point based on the current interconnection state; the state update unit is used to update the current interconnection state based on the edge prediction information of the current point.

[0087] In some embodiments, the intrinsic information of the point cloud data to be predicted includes the intrinsic information of each point in the point cloud data to be predicted; the edge prediction information of the point cloud data to be predicted includes the edge prediction information of each point in the point cloud data to be predicted; the prediction module 520 is further configured to input the intrinsic information of each point in the point cloud data to be predicted into a pre-trained edge prediction model in a first preset order to obtain the edge prediction information of each point in the point cloud data to be predicted.

[0088] In some embodiments, the first preset order is obtained by sorting each point in the point cloud data to be predicted using a traversal algorithm; the feature extraction module 510 is also used to extract features from each point in the point cloud data to be predicted according to the first preset order based on a pre-trained encoder, so as to obtain the intrinsic information of each point in the point cloud data to be predicted.

[0089] In some embodiments, the state update unit includes a first long short-term memory network; the edge prediction unit includes a second long short-term memory network; the prediction module 520 is further configured to repeatedly perform the following steps until the edge prediction information of each point in the point cloud data to be predicted is obtained: determining the current point from the point cloud data to be predicted in a first preset order; inputting the intrinsic information of the current point into the second long short-term memory network according to the current interconnection state in the first long short-term memory network to obtain the edge prediction information of the current point; updating the current interconnection state stored in the first long short-term memory network according to the current point and the edge prediction information of the current point.

[0090] In some embodiments, the intrinsic information of the current point includes spatial feature information; the second long short-term memory network is used to evaluate the spatial correlation between the current point and the current interconnection state based on the spatial feature information of the current point, and to determine the edge prediction information of the current point based on the spatial correlation.

[0091] In some embodiments, the point cloud data gridding device further includes a preprocessing module 540. The preprocessing module 540 is used to filter and denoise the raw point cloud data to obtain effective point cloud data; and to standardize the coordinate information of the effective point cloud data to obtain point cloud data to be predicted.

[0092] In some embodiments, the pre-trained encoder is obtained by training the autoencoder based on the first training point cloud data; the autoencoder includes an encoder to be trained and a decoder; the encoder to be trained is used to extract features from the first training point cloud data to obtain first training intrinsic information; the decoder is used to reconstruct the first training point cloud data based on the first training intrinsic information output by the encoder to be trained, with the training objective of minimizing the difference between the first training point cloud data and the reconstructed first training point cloud data.

[0093] In some embodiments, the point cloud data gridding device further includes an optimization module 550. The optimization module 550 is used to optimize the gridded data to obtain optimized gridded data.

[0094] In some embodiments, the optimization module 550 is further configured to smooth the gridded data to obtain intermediate gridded data; and to perform detail enhancement processing on the intermediate gridded data to obtain optimized gridded data.

[0095] It should be noted that the point cloud data meshing device provided in this embodiment includes modules (or units) for performing the various steps of the point cloud data meshing method in the above embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0096] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0097] like Figure 6 As shown, the electronic device may include a processor 601 and a memory 602 storing computer program instructions.

[0098] Specifically, the processor 601 may include a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of the present invention.

[0099] Memory 602 may include mass storage for data or instructions. For example, and not limitingly, memory 602 may include a hard disk drive (HDD), floppy disk drive, flash memory, optical disk, magneto-optical disk, magnetic tape, or Universal Serial Bus (USB) drive, or a combination of two or more of these. In one instance, memory 602 may include removable or non-removable (or fixed) media, or memory 602 may be non-volatile solid-state memory. Memory 602 may be internal or external to the integrated gateway disaster recovery device.

[0100] In one instance, memory 602 may be read-only memory (ROM). In one instance, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or flash memory, or a combination of two or more of these.

[0101] Memory 602 may include read-only memory (ROM), random access memory (RAM), disk storage media device, optical storage media device, flash memory device, electrical, optical, or other physical / tangible memory storage device. Therefore, generally, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to one aspect of this disclosure.

[0102] The processor 601 implements the point cloud data meshing method in the above embodiments by reading and executing computer program instructions stored in the memory 602.

[0103] In one example, the electronic device may also include a communication interface 603 and a bus 604. Wherein, as... Figure 6 As shown, the processor 601, memory 602, and communication interface 603 are connected through bus 604 and complete communication with each other.

[0104] The communication interface 603 is mainly used to realize communication between various modules, devices, units and / or equipment in the embodiments of the present invention.

[0105] Bus 604 includes hardware, software, or both, that couples components of an online data traffic metering device together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 304 may include one or more buses. While specific buses are described and illustrated in embodiments of the invention, the invention contemplates any suitable bus or interconnect.

[0106] Furthermore, in conjunction with the methods in the above embodiments, this invention can be implemented using a computer storage medium. This computer storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the point cloud data meshing methods described in the above embodiments.

[0107] This application also provides a computer program product, including a computer program that, when executed, implements any of the point cloud data meshing methods described in the above embodiments.

[0108] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0109] The functional blocks shown in the above-described structural diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the required tasks. Programs or code segments can be stored on a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, read-only memory (ROM), flash memory, erasable read-only memory (EROM), floppy disks, compact disc read-only memory (CD-ROM), optical disks, hard disks, fiber optic media, radio frequency (RF) links, etc. Code segments can be downloaded via computer networks such as the Internet, intranets, etc.

[0110] It should also be noted that the exemplary embodiments mentioned in this invention describe methods or systems based on a series of steps or apparatus. However, this invention is not limited to the order of the steps described above; that is, the steps can be performed in the order mentioned in the embodiments, or in a different order, or several steps can be performed simultaneously.

[0111] The aspects of this disclosure have been described above with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block in the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that these instructions, executable via the processor of the computer or other programmable data processing apparatus, enable the implementation of the functions / actions specified in one or more blocks of the flowchart illustrations and / or block diagrams. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field-programmable logic circuit. It is also understood that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can also be implemented by special-purpose hardware performing the specified functions or actions, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0112] The above description is merely a specific embodiment of the present invention. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, modules, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the protection scope of the present invention.

Claims

1. A method for meshing point cloud data, characterized in that, include: Based on the pre-trained encoder, feature extraction is performed on the point cloud data to be predicted to obtain the intrinsic information of the point cloud data to be predicted. The intrinsic information of the point cloud data to be predicted is input into a pre-trained edge prediction model to obtain the edge prediction information of the point cloud data to be predicted. Based on the edge prediction information of the point cloud data to be predicted, gridded data corresponding to the point cloud data to be predicted is generated.

2. The method according to claim 1, characterized in that, The pre-trained edge prediction model includes a state update unit and an edge prediction unit, used to predict the edge prediction information of the point cloud data to be predicted sequentially based on the intrinsic information of the point cloud data to be predicted, according to the state update unit and the edge prediction unit; the edge prediction unit is used to predict the edge prediction information of the current point based on the current interconnection state; the state update unit is used to update the current interconnection state based on the edge prediction information of the current point.

3. The method according to claim 2, characterized in that, The intrinsic information of the point cloud data to be predicted includes the intrinsic information of each point in the point cloud data to be predicted; the edge prediction information of the point cloud data to be predicted includes the edge prediction information of each point in the point cloud data to be predicted. The step of inputting the intrinsic information of the point cloud data to be predicted into a pre-trained side prediction model to obtain the side prediction information of the point cloud data to be predicted includes: According to the first preset order, the intrinsic information of each point in the point cloud data to be predicted is input into the pre-trained edge prediction model to obtain the edge prediction information of each point in the point cloud data to be predicted.

4. The method according to claim 3, characterized in that, The first preset order is obtained by sorting each point in the point cloud data to be predicted using a traversal algorithm; the feature extraction of the point cloud data to be predicted based on the pre-trained encoder to obtain the intrinsic information of the point cloud data to be predicted includes: Based on the pre-trained encoder, features are extracted from each point in the point cloud data to be predicted in a first preset order to obtain the intrinsic information of each point in the point cloud data to be predicted.

5. The method according to claim 3, characterized in that, The state update unit includes a first long short-term memory network; the edge prediction unit includes a second long short-term memory network; the step of inputting the intrinsic information of each point in the point cloud data to be predicted into the pre-trained edge prediction model according to the first preset order to obtain the edge prediction information of each point in the point cloud data to be predicted includes: Repeat the following steps until the edge prediction information for each point in the point cloud data to be predicted is obtained: The current point is determined from the point cloud data to be predicted according to the first preset order; Based on the current interconnection state in the first long short-term memory network, the intrinsic information of the current point is input into the second long short-term memory network to obtain the edge prediction information of the current point; The current interconnection state stored in the first long short-term memory network is updated based on the current point and the edge prediction information of the current point.

6. The method according to claim 5, characterized in that, The intrinsic information of the current point includes spatial feature information; the second long short-term memory network is used to evaluate the spatial correlation between the current point and the current interconnection state based on the spatial feature information of the current point, and to determine the edge prediction information of the current point based on the spatial correlation.

7. The method according to claim 1, characterized in that, Before the pre-trained encoder performs feature extraction on the point cloud data to be predicted to obtain the intrinsic information of the point cloud data to be predicted, the method further includes: The raw point cloud data is filtered and denoised to obtain effective point cloud data; The coordinate information of the effective point cloud data is standardized to obtain the point cloud data to be predicted.

8. The method according to claim 1, characterized in that, The pre-trained encoder is obtained by training the autoencoder based on the first training point cloud data; the autoencoder includes an encoder to be trained and a decoder; the encoder to be trained is used to extract features from the first training point cloud data to obtain first training intrinsic information. The decoder is used to reconstruct the first training point cloud data based on the first training intrinsic information output by the encoder to be trained, with the training objective of minimizing the difference between the first training point cloud data and the reconstructed first training point cloud data.

9. The method according to any one of claims 1-8, characterized in that, The method further includes: The gridded data is then optimized to obtain optimized gridded data.

10. The method according to claim 9, characterized in that, The optimization process for the gridded data to obtain optimized gridded data includes: The gridded data is smoothed to obtain intermediate gridded data; The intermediate gridded data is subjected to detail enhancement processing to obtain the optimized gridded data.

11. A point cloud data gridding device, characterized in that, The device includes: The feature extraction module is used to extract features from the point cloud data to be predicted based on the pre-trained encoder, so as to obtain the intrinsic information of the point cloud data to be predicted. The prediction module is used to input the intrinsic information of the point cloud data to be predicted into a pre-trained side prediction model to obtain the side prediction information of the point cloud data to be predicted. The generation module is used to generate gridded data corresponding to the point cloud data to be predicted based on the edge prediction information of the point cloud data to be predicted.

12. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the point cloud data meshing method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions, which, when executed by a processor, implement the point cloud data meshing method as described in any one of claims 1-10.

14. A computer program product, characterized in that, Includes a computer program, which, when executed, implements the point cloud data meshing method as described in any one of claims 1-10.

Images