Method, system and device for generating polygonal mesh on point cloud and storage medium

By using a method to generate polygonal meshes, and combining point sets and mesh generation algorithms with a mesh diffusion probability model and a 3D graph attention network, the problem of insufficient flexibility and universality of mesh shapes in existing technologies is solved, and more accurate and aesthetically pleasing 3D shape generation is achieved.

CN118967975APending Publication Date: 2024-11-15HARBIN INSTITUTE OF TECHNOLOGY (SHENZHEN) (INSTITUTE OF SCIENCE AND TECHNOLOGY INNOVATION HARBIN INSTITUTE OF TECHNOLOGY SHENZHEN)
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202411065082.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-05
Publication Date
2024-11-15

AI Technical Summary

Technical Problem

Existing 3D shape generation technologies mainly focus on generating triangular or quadrilateral meshes from point cloud data, which limits the flexibility and universality of mesh shapes, resulting in unsatisfactory generation effects.

Method used

By acquiring the point set of the target product, a polygonal mesh is generated using a mesh generation algorithm. The edge set length is varied according to the number of other vertices associated with each vertex to construct the polygonal mesh. A mesh diffusion probability model and a 3D graph attention network are used to improve the generation quality.

Benefits of technology

It enables the generation of polygonal meshes of various shapes, improving the flexibility and universality of mesh shapes, and making the 3D shape generation effect more accurate and beautiful.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118967975A_ABST
    Figure CN118967975A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of point cloud grid processing. The invention discloses a method, a system and equipment for generating a polygonal mesh on a point cloud and a storage medium. The method, the system and the equipment can improve the flexibility and the universality of a generated mesh shape. The method for generating the polygonal mesh on the point cloud comprises the following steps: acquiring a point set of a target product, wherein the point set comprises a plurality of vertexes; the point set is input into a grid generation algorithm, a first edge set corresponding to each vertex in the point set is generated according to the grid generation algorithm, a first to-be-formed grid is generated according to a first face set formed by all the first edge sets, and the length of the first edge set corresponding to each vertex is equal to the length of the first face set corresponding to each vertex. The vertex changes according to the number of other vertexes associated with the vertex; and obtaining a polygonal mesh based on the first to-be-formed mesh, wherein the polygonal mesh is used for presenting a three-dimensional shape structure.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of point cloud mesh processing, and more specifically, to a method, system, device and storage medium for generating polygonal meshes on point clouds. Background Art

[0002] The application of three-dimensional shape generation technology in computer graphics, virtual reality or autonomous driving is very extensive. Existing three-dimensional shape generation technology mainly focuses on the generation of point cloud data (the point cloud data described in this article can be understood as a point set), and generates triangular meshes or quadrilateral meshes based on point cloud data. For example, the Chinese invention patent application with publication number CN102306397A discloses a method for meshing point cloud data, which is to determine the normal vector, main direction and singular point information of each point of the point cloud data from the point cloud data; then the point cloud is parameterized by the coordinates of each point of the point cloud data, the main direction of the point, the normal vector and the singular point information; then, a triangular mesh consistent with the shape of the point cloud data is established based on the point cloud parameterization result; then, a quadrilateral mesh is established on the basis of the established triangular mesh. The specific steps of establishing a quadrilateral mesh include obtaining the isoline segments in each triangle; obtaining the intersection points between the isoline segments; for each intersection point between the isoline segments, the adjacent points of the intersection are found and connected according to the connection relationship between the isoline segments, and a quadrilateral mesh with uniform shape and direction consistent with the main direction is established. However, this method can only be used to generate triangular meshes or quadrilateral meshes, which limits the flexibility and universality of the generated mesh shapes, resulting in unsatisfactory generation of three-dimensional shapes. Therefore, the mesh generation problem has always been a challenging and urgent problem to be solved. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method, system, device and storage medium for generating a polygonal mesh on a point cloud, which can improve the flexibility and universality of generating mesh shapes. The embodiments of the present application are mainly implemented through the following technical solutions:

[0004] According to a first aspect of an embodiment of the present application, there is provided a method for generating a polygonal mesh on a point cloud, comprising:

[0005] Acquire a point set of a target product, wherein the point set includes a plurality of vertices;

[0006] Inputting the point set into a mesh generation algorithm, generating a corresponding first edge set for each vertex in the point set according to the mesh generation algorithm, and generating a first to-be-formed mesh according to a first face set formed by all the first edge sets, wherein the length of the first edge set corresponding to each vertex varies according to the number of other vertices associated with the vertex;

[0007] A polygonal mesh is obtained based on the first mesh to be formed, and the polygonal mesh is used to present a three-dimensional shape structure.

[0008] According to one embodiment of the present application, the step of obtaining a point set of a target product, wherein the point set includes a plurality of vertices, comprises:

[0009] The target product is processed based on a point cloud generation model to generate the point set, wherein the point cloud generation model is a model of a three-dimensional point set that captures and records point data on the surface of the target product through a sensor device to generate a three-dimensional point set that expresses the surface shape and characteristics of the target product.

[0010] According to an embodiment of the present application, the step of inputting the point set into a mesh generation algorithm, generating a corresponding first edge set for each vertex in the point set according to the mesh generation algorithm, and generating a first to-be-formed mesh according to a first face set composed of all first edge sets, wherein the length of the first edge set corresponding to each vertex varies according to the number of other vertices associated with the vertex, comprises:

[0011] Setting a current vertex variable and a preset vertex variable, wherein the current vertex variable is used to point to one of the vertices in the point set, and the preset vertex variable is initialized to point to the same vertex as the current vertex variable;

[0012] Set the last sampling vertex variable, the starting vertex variable, the target sampling vertex variable, the second face set, the second edge set, the end mark vertex variable and the start mark vertex variable, wherein the last sampling vertex variable, the start vertex variable and the target sampling vertex variable are all initialized to NULL, the last sampling vertex variable, the start vertex variable and the target sampling vertex variable are all used to point to one of the vertices in the point set, and the vertex pointed to by the target sampling vertex variable is not the vertex pointed to by the current vertex variable; the second face set and the second edge set are both initialized to empty sets; the end mark vertex variable is used to point to the vertex indicating the end of the prediction of all faces; the start mark vertex variable is used to point to the vertex indicating the end of the prediction of one face and the beginning of the prediction of the next face; the vertices pointed to by the end mark vertex variable and the start mark vertex variable belong to the point set;

[0013] forming a second mesh to be formed based on the point set and the second surface set;

[0014] The loop process is executed until the preset condition is met; the loop process includes:

[0015] Extracting a sub-mesh from the second mesh to be formed, wherein the sub-mesh includes the vertex pointed to by the current vertex variable;

[0016] modifying the target sampling vertex variables based on the sub-meshes and a mesh diffusion probability model of the mesh generation algorithm;

[0017] In the case that the vertex pointed to by the target sampling vertex variable and the vertex pointed to by the previous sampling vertex variable are not the vertex pointed to by the start mark vertex variable, generating edges corresponding to the target sampling vertex variable and the previous sampling vertex variable, and adding them to the second edge set;

[0018] When the vertex pointed to by the target sampling vertex variable is not the vertex pointed to by the start mark vertex variable, and the vertex pointed to by the previous sampling vertex variable is the vertex pointed to by the start mark vertex variable, replace the vertex pointed to by the start vertex variable with the vertex pointed to by the target sampling vertex variable;

[0019] In a case where the vertex pointed to by the target sampling vertex variable is the vertex pointed to by the starting marking vertex variable, generating a third mesh to be formed based on the point set, the previous sampling vertex variable, the starting vertex variable, the second edge set and the second face set;

[0020] Replace the vertex pointed to by the previous sampling vertex variable with the vertex pointed to by the target sampling vertex variable;

[0021] Replace the vertex pointed to by the current vertex variable with the vertex pointed to by the target sampling vertex variable;

[0022] The preset condition is that the vertex pointed to by the current vertex variable is the vertex pointed to by the end mark vertex variable.

[0023] According to an embodiment of the present application, the step of modifying the target sampling vertex variable based on the sub-grid and the mesh diffusion probability model of the mesh generation algorithm includes:

[0024] Inputting the sub-grid into the grid diffusion probability model to obtain a reconstructed sub-grid;

[0025] Removing vertices and edges shared with the sub-mesh from the reconstructed sub-mesh to obtain a target vertex;

[0026] Replace the vertex pointed to by the target sampling vertex variable with the target vertex.

[0027] According to an embodiment of the present application, the reconstructed sub-mesh has one more vertex and one more edge than the sub-mesh in structure.

[0028] According to one embodiment of the present application, the step of generating a third mesh to be formed based on the point set, the last sampled vertex variable, the starting vertex variable, the second edge set and the second face set comprises:

[0029] Based on the last sampled vertex variable and the starting vertex variable, generate edges corresponding to the last sampled vertex variable and the starting vertex variable, and add the second edge set to obtain the first edge set corresponding to the vertex pointed to by the preset vertex variable;

[0030] Adding the first edge set to the second face set to obtain a third face set;

[0031] Assign the second edge set to an empty set;

[0032] Replace the vertex pointed to by the preset vertex variable with the vertex pointed to by the target sampling vertex variable;

[0033] Constructing the third to-be-formed mesh based on the point set and the third face set;

[0034] Replacing the value corresponding to the second grid to be formed with the value corresponding to the third grid to be formed;

[0035] In a case where the vertex pointed to by the target sampling vertex variable is the vertex pointed to by the end marker vertex variable, the third face set is used as the first face set, and the third mesh to be formed is used as the first mesh to be formed.

[0036] According to one embodiment of the present application, the training method of the grid diffusion probability model includes:

[0037] Acquire a training set, wherein each training data in the training set is a sub-grid;

[0038] Inputting target training data into the three-dimensional graph attention network and the forward diffusion network in the grid diffusion probability model to obtain a completely shielded grid, wherein the target training data is any training data in the training set;

[0039] Inputting the completely shielded mesh into the three-dimensional graph attention network and the reverse denoising network in the mesh diffusion probability model to obtain a reconstructed mesh;

[0040] Calculating a first loss function of the inverse denoising network based on the reconstructed grid, and adjusting model parameters of the inverse denoising network based on the first loss function;

[0041] A second loss function of the forward diffusion network is calculated based on the fully shielded grid and the first loss function, and model parameters of the forward diffusion network are adjusted based on the second loss function.

[0042] According to a second aspect of an embodiment of the present application, a system for generating a polygonal mesh on a point cloud is provided. The system for generating a polygonal mesh on a point cloud comprises:

[0043] A point set acquisition module, used to acquire a point set of a target product, wherein the point set includes a plurality of vertices;

[0044] A first mesh generation module to be formed is used to input the point set into a mesh generation algorithm, generate a corresponding first edge set for each vertex in the point set according to the mesh generation algorithm, and generate a first mesh to be formed according to a first face set composed of all the first edge sets, wherein the length of the first edge set corresponding to each vertex varies according to the number of other vertices associated with the vertex;

[0045] A polygonal mesh forming module is used to obtain a polygonal mesh based on the first mesh to be formed, and the polygonal mesh is used to present a three-dimensional shape structure.

[0046] According to a third aspect of an embodiment of the present application, a terminal device is provided, including a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing the steps of the method for generating a polygonal mesh on a point cloud provided in the first aspect of the embodiment of the present application.

[0047] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided for storing a computer program, wherein the computer program enables a computer to execute the steps of the method for generating a polygonal mesh on a point cloud provided in the first aspect of the embodiment of the present application.

[0048] The beneficial effects of the embodiments of the present application include:

[0049] The embodiment of the present application is to conceive the process of mesh generation as the evolution process of points, lines and surfaces. Specifically, a point set of a target product is obtained, and the point set contains multiple vertices; the point set is input into a mesh generation algorithm, and a corresponding first edge set is generated for each vertex in the point set according to the mesh generation algorithm, and a first mesh to be formed is generated according to a first face set composed of all first edge sets, wherein the length of the first edge set corresponding to each vertex varies according to the number of other vertices associated with this vertex; a polygonal mesh is obtained based on the first mesh to be formed, and the polygonal mesh is used to present a three-dimensional shape structure. Since the length of the edge set corresponding to each vertex varies according to the number of other vertices associated with this vertex, edge sets of different lengths can form faces of different shapes, and then polygonal meshes are formed by faces of different shapes. Therefore, compared with the prior art that can only generate triangular meshes or quadrilateral meshes, the embodiment of the present application can generate polygonal meshes of various shapes, thereby improving the flexibility and universality of the generated mesh shape, making the generation effect of the three-dimensional shape more accurate and beautiful. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0051] Figure 1 A flowchart of a method for generating a polygonal mesh on a point cloud according to some embodiments of the present invention;

[0052] Figure 2 Schematic diagram of forward diffusion in the present invention;

[0053] Figure 3 A schematic diagram of reverse denoising in the present invention;

[0054] Figure 4 Schematic diagram of the working principle of the three-dimensional graph attention network in the present invention;

[0055] Figure 5 A reference diagram of the visualization result of the polygonal mesh of the present invention;

[0056] Figure 6 A structural block diagram of a system for generating a polygonal mesh on a point cloud according to some embodiments of the present invention;

[0057] Figure 7 It is a principle block diagram of the terminal device of the present invention in some embodiments. DETAILED DESCRIPTION

[0058] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0059] It should be noted that, in this article, the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0060] The terms "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0061] The terms "comprises," "includes," or any other variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or elements is not necessarily limited to those steps or elements explicitly listed but may include other steps or elements not explicitly listed or inherent to such process, method, product, or apparatus.

[0062] Unless otherwise defined, all technical and scientific terms used in the specification of this application have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in the specification of this application includes any and all combinations of one or more related listed items.

[0063] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0064] like Figure 1 FIG. 1 is a flowchart of a method for generating a polygonal mesh on a point cloud provided by an embodiment of the present application. Figure 1 In the method of generating a polygonal mesh on a point cloud, the method comprises:

[0065] S1. Obtain a point set of a target product, wherein the point set includes a plurality of vertices.

[0066] The target product may be any item, such as a model airplane, a desk lamp, a table, a fan, assembly line machinery or other physical objects.

[0067] The point set refers to a set of vectors in a three-dimensional coordinate system, that is, point cloud data.

[0068] Each of the vertices includes three-dimensional coordinates, and some may contain color information or reflection intensity information.

[0069] Furthermore, the S1 step includes:

[0070] S11. Process the target product based on a point cloud generation model to generate the point set, wherein the point cloud generation model is a model of a three-dimensional point set that captures and records point data on the surface of the target product through a sensor device to generate a three-dimensional point set that expresses the surface shape and characteristics of the target product.

[0071] The sensor device is a laser scanner or a stereo camera system.

[0072] In some embodiments, the point cloud generation model may also be a point cloud generation model based on deep learning, which is also capable of processing complex point cloud data and generating high-quality three-dimensional models.

[0073] In other implementations, those skilled in the art may select a corresponding point cloud generation model according to actual needs, and this document does not impose any further limitations on this.

[0074] S2. Input the point set into a mesh generation algorithm, generate a corresponding first edge set for each vertex in the point set according to the mesh generation algorithm, and generate a first mesh to be formed according to a first face set composed of all first edge sets, wherein the length of the first edge set corresponding to each vertex varies according to the number of other vertices associated with this vertex.

[0075] The length of the first edge set refers to the number of elements in the first edge set.

[0076] Furthermore, the S2 step includes:

[0077] S21. Set a current vertex variable and a preset vertex variable, wherein the current vertex variable is used to point to one of the vertices in the point set, and the preset vertex variable is initialized to point to the same vertex as the current vertex variable.

[0078] It should be noted that the current vertex variable is used to calculate the next vertex. The next vertex is used to generate one of the edges in the first edge set, and the first edge set corresponds to the vertex pointed to by the preset vertex variable.

[0079] S22, setting the previous sampling vertex variable, the starting vertex variable, the target sampling vertex variable, the second face set, the second edge set, the end mark vertex variable and the start mark vertex variable, wherein the previous sampling vertex variable, the starting vertex variable and the target sampling vertex variable are all initialized to NULL, the previous sampling vertex variable, the starting vertex variable and the target sampling vertex variable are all used to point to one of the vertices in the point set, and the vertex pointed to by the target sampling vertex variable is not the vertex pointed to by the current vertex variable; the second face set and the second edge set are both initialized to empty sets; the end mark vertex variable is used to point to the vertex indicating the end of prediction of all faces; the start mark vertex variable is used to point to the vertex indicating the end of prediction of one face and the start of prediction of the next face; the vertices pointed to by the end mark vertex variable and the start mark vertex variable both belong to the point set.

[0080] S23. Construct a second grid to be formed based on the point set and the second surface set.

[0081] It should be understood that in step S23, the second face set is an empty set. Therefore, the second mesh to be formed now only has a plurality of vertices in the point set.

[0082] S24, executing a loop process until a preset condition is satisfied, wherein the preset condition is that the vertex pointed to by the current vertex variable is the vertex pointed to by the end mark vertex variable; the loop process includes:

[0083] S241. Extract a sub-mesh from the second mesh to be formed, where the sub-mesh includes the vertex pointed to by the current vertex variable.

[0084] It should be understood that the value corresponding to the second grid to be formed changes dynamically with the cycle. Exemplarily, in the first cycle, the value corresponding to the second grid to be formed is the point set and the second face set which is an empty set; in the nth cycle, the value corresponding to the second grid to be formed is the point set and the third face set, and the third face set is actually the second face set which is not an empty set.

[0085] S242: based on the sub-grid and the mesh diffusion probability model of the mesh generation algorithm, modify the target sampling vertex variable. The application of the mesh diffusion probability model can ensure the equivariance and robustness of the point set, improve the accuracy of the target sampling vertex variable, and thus further improve the generation quality of the polygonal mesh.

[0086] Furthermore, the step S242 includes:

[0087] S2421. Input the sub-grid into the grid diffusion probability model to obtain a reconstructed sub-grid.

[0088] The reconstructed sub-mesh has one more vertex and one more edge than the sub-mesh in structure.

[0089] S2422: Remove vertices and edges shared with the sub-mesh from the reconstructed sub-mesh to obtain a target vertex.

[0090] S2423. Replace the vertex pointed to by the target sampling vertex variable with the target vertex.

[0091] S243. When the vertex pointed to by the target sampling vertex variable and the vertex pointed to by the previous sampling vertex variable are not the vertex pointed to by the start mark vertex variable, generate edges corresponding to the target sampling vertex variable and the previous sampling vertex variable, and add them to the second edge set.

[0092] In the embodiments of the present application, the generation of an edge refers to a line formed by connecting two vertices.

[0093] S244: When the vertex pointed to by the target sampling vertex variable is not the vertex pointed to by the start mark vertex variable, and the vertex pointed to by the previous sampling vertex variable is the vertex pointed to by the start mark vertex variable, replace the vertex pointed to by the start vertex variable with the vertex pointed to by the target sampling vertex variable.

[0094] S245. When the vertex pointed to by the target sampling vertex variable is the vertex pointed to by the starting mark vertex variable, generate a third mesh to be formed based on the point set, the previous sampling vertex variable, the starting vertex variable, the second edge set and the second face set.

[0095] Furthermore, the step S245 includes:

[0096] S2451. Based on the previous sampled vertex variable and the starting vertex variable, generate edges corresponding to the previous sampled vertex variable and the starting vertex variable, and add the second edge set to obtain the first edge set corresponding to the vertex pointed to by the preset vertex variable.

[0097] In step S2451, it indicates that the first edge set corresponding to one of the vertices in the point set (that is, the vertex pointed to by the preset vertex variable) has been completely generated, and a face generated by the first edge set can be added to the second face set.

[0098] S2452. Add the first edge set to the second face set to obtain a third face set.

[0099] S2453. Assign the second edge set to an empty set.

[0100] The purpose of setting the S2453 step is to collect all edges corresponding to the next vertex.

[0101] S2454: Replace the vertex pointed to by the preset vertex variable with the vertex pointed to by the target sampling vertex variable.

[0102] The step S2454 means that the next face to be generated is the face corresponding to the vertex pointed to by the target sampling vertex variable.

[0103] S2455. Construct the third grid to be formed based on the point set and the third face set.

[0104] S2456. Replace the value corresponding to the second grid to be formed with the value corresponding to the third grid to be formed.

[0105] The setting of step S2456 can make the value corresponding to the second grid to be formed change dynamically with the cycle.

[0106] S2457. When the vertex pointed to by the target sampling vertex variable is the vertex pointed to by the end marker vertex variable, use the third face set as the first face set and use the third mesh to be formed as the first mesh to be formed.

[0107] S246: Replace the vertex pointed to by the previous sampling vertex variable with the vertex pointed to by the target sampling vertex variable.

[0108] S247: Replace the vertex pointed to by the current vertex variable with the vertex pointed to by the target sampling vertex variable.

[0109] S3. Obtain a polygonal mesh based on the first mesh to be formed, wherein the polygonal mesh is used to present a three-dimensional shape structure.

[0110] Through the above technical solution, since the length of the edge set corresponding to each vertex in the present application will change according to the number of other vertices associated with this vertex, edge sets of different lengths can form faces of different shapes, and then the faces of different shapes form polygonal meshes. Therefore, compared with the existing technology that can only generate triangular meshes or quadrilateral meshes, the embodiment of the present application can generate polygonal meshes of various shapes, thereby improving the flexibility and universality of the generated mesh shape, making the generation effect of the three-dimensional shape more accurate and beautiful.

[0111] In some embodiments, the training method of the grid diffusion probability model includes:

[0112] S41. Obtain a training set, wherein each training data in the training set is a sub-grid.

[0113] S42, inputting the target training data into the three-dimensional graph attention network and the forward diffusion network in the mesh diffusion probability model to obtain a completely shielded mesh, wherein the target training data is any training data in the training set. The use of the three-dimensional graph attention network can not only enhance the ability to extract mesh structure features and improve the generation quality of polygonal meshes, but also reduce the complexity of calculating the next predicted vertex.

[0114] The three-dimensional graph attention network (3D-GAT) is an application of the graph attention network (GAT) on three-dimensional data or graph structure data. The three-dimensional graph attention network combines the ability of the graph neural network to process graph structure data and the advantages of the attention mechanism in feature selection. It is suitable for processing three-dimensional data with complex spatial structures and inter-vertex relationships, such as three-dimensional point clouds, three-dimensional meshes, etc.

[0115] The forward diffusion network is a noise adding process. In the embodiment of the present application, the noise adding process is understood as a masking process for each vertex and each edge in the target training data (a sub-grid).

[0116] The step S42 can be understood as inputting the target training data into the three-dimensional graph attention network, and the three-dimensional graph attention network outputs the score of each vertex in the target training data, and determines the probability of each vertex being predicted next based on these scores. This application selects the vertex with a high probability as the next predicted vertex. The forward diffusion network performs a shielding operation on this "next predicted vertex" and simultaneously shields the edge corresponding to the "next predicted vertex" and the current vertex.

[0117] Specifically, in the embodiment of the present application, each vertex has a certain probability of transforming from a live state to a shielded state, and in each diffusion step t, only one vertex is shielded, and the edge corresponding to the vertex and the previous shielded vertex is shielded. When the sampled vertex is the vertex pointed to by the start marker vertex variable, it means that a face has been shielded, and the model activates all vertices. After N time steps, all edges will be in a shielded state. This shielding process is similar to a vertex sorting random walk process.

[0118] In each diffusion step, the forward diffusion network will sample a vertex and mask it. If the previous masked vertex is not the vertex pointed to by the start mark vertex variable, the edge between the current vertex and the previous vertex will also be masked. When the vertex pointed to by the start mark vertex variable is sampled, it indicates that the noise masking of a face is currently completed, and all the masked vertices in this face will be activated. For example, in Figure 2 In the first diffusion step, the vertex labeled 1 is shielded; in the second diffusion step, the vertex labeled 2 and the edge labeled a are shielded; in the third diffusion step, the vertex labeled 3 and the edge labeled b are shielded; in the fourth diffusion step, the vertex pointed to by the start marker vertex variable is sampled, and the edge labeled c is shielded. At this time, it indicates that the noise shielding of a face is currently completed, and all the shielded vertices in this face are activated, that is, the vertices labeled 1, 2 and 3 are activated. After a series of diffusion steps, all edges are in a shielded state. At this time, the target training data is equivalent to an empty grid with only vertices but no edges and faces (that is, a completely shielded grid). This paper defines this state as the final state of the forward diffusion of the mesh diffusion probability model (MeshDG).

[0119] In order to realize the above forward diffusion process, the present application designs a parameterized neural network q(σ|M0), where σ represents an indefinite length arrangement order of all given vertices, ending with the vertex pointed to by the end marker vertex variable; M0 represents the target training data at time 0, that is, the grid without noise. The present application expresses the calculation formula of q(σ|M0) as:

[0120] q(σ|M0)=∏ t q(σ t |M0,σ <t );

[0121] Among them, σ represents an indefinite length arrangement order of all given vertices, ending with the vertex pointed to by the end marker vertex variable; M0 represents the target training data at time 0; t represents the tth sampled shielded vertex; σ t is a sequence sampled given the original mesh (i.e., target training data) and the generated vertex sequence σ.

[0122] This application uses a three-dimensional graph attention network to encode the structural information in the target training data, and adds the position encoding of the vertex to the vertex feature. After encoding by the three-dimensional graph attention network, the vertex v i The embedding of i Let V′ be the set of vertices that may be sampled in the next step. The classification distribution of the parameterized neural network fitting can be expressed as the following formula:

[0123]

[0124] Among them, σ represents an indefinite length arrangement order of all given vertices, ending with the vertex pointed to by the end marker vertex variable; M0 represents the target training data at time 0; t represents the tth sampled shielded vertex; σ t is the sequence sampled given the original mesh (i.e., the target training data) and the generated vertex sequence σ, h i Represents vertex v i embedding, V′ represents the set of vertices that may be sampled in the next step, i′ represents one of the vertices in V′, and h i′ represents the embedding of vertex i′, and the exp() function represents the natural exponential function.

[0125] S43. Input the completely shielded grid into the three-dimensional graph attention network and the reverse denoising network in the grid diffusion probability model to obtain a reconstructed grid.

[0126] The inverse denoising network is a denoising process.

[0127] The reconstructed mesh will have one more edge and one more vertex than the target training data.

[0128] The embodiment of the present application utilizes the three-dimensional graph attention network for message passing and embedding coding. However, due to the huge number of vertices contained in the completely shielded mesh, encoding the entire completely shielded mesh will result in huge computational overhead. Therefore, in each generation step, the present application only retains the vertices directly associated with the vertex that currently needs to be denoised, and ignores other unrelated vertices. The reverse denoising network uses an embedding representation layer to encode each vertex v i Encoded into a continuous latent space, the specific formula is as follows:

[0129]

[0130] Among them, h i Represents vertex ν i embedding, W is the weight matrix, h j Represents vertex ν j The embedding of The jth vertex ν j , Represents vertex v i The neighbor set of α i,j Represents vertex v i and vertex v j The dependency weight of .

[0131] Furthermore, α i,j The calculation formula is as follows:

[0132]

[0133] Among them, the exp() function represents the natural exponential function, LeakyReLU() represents the activation function used in deep learning, and a T Represents the self-attention vector, which is used to capture the dependencies between different vertices in the sampled sequence, that is, the current vertex v i Dependence on the previously predicted vertex sequence; W is the weight matrix; h i Represents vertex v i Embedding of j Represents vertex v j Embedding Represents vertex v i The neighbor set of The kth vertex v in k ;h k Represents the embedding of vertex vk.

[0134] In this application, the reverse denoising process can be referred to Figure 3 shown.

[0135] exist Figure 3 , assuming that the current active vertex is labeled 3, the next step of the mesh diffusion probability model will predict an edge associated with the vertex labeled 3. At this time, the mesh diffusion probability model will intercept the one-hop vertices associated with vertex 3, that is, vertices 1, 2, and 4 in the graph, plus the active vertex 3 itself to form a sub-mesh structure. Then, the sub-mesh structure is input into the three-dimensional graph attention network for embedded representation and feature extraction. Through the three-dimensional graph attention network, the mesh diffusion probability model can learn the relationship and dependency between vertices, thereby predicting that the next vertex is labeled 1, and activating the corresponding edges of labels 1 and 3. This process will be repeated until the mesh diffusion probability model predicts that the next active vertex is the vertex pointed to by the end marker vertex variable.

[0136] refer to Figure 4 , the 3D graph attention network passes the current input mesh data through message passing and aggregation, and then pays attention to each vertex (including the vertex pointed to by the start mark vertex variable and the vertex pointed to by the end mark vertex variable, that is, the vertex pointed to by the start mark vertex variable). Figure 4 The start token in the end token vertex variable points to the vertex, that is Figure 4The stop token in the grid is used for embedding representation. The data is processed by the attention mechanism to capture the interdependence between vertices and give a score to each vertex. Then, after being processed by the Softmax function, the corresponding probability distribution can be obtained. This probability distribution is used to reflect the possibility of each vertex becoming the next active point. Then, the grid diffusion probability model selects the vertex with the highest probability as the next active vertex.

[0137] S44. Calculate a first loss function of the inverse denoising network based on the reconstructed grid, and adjust model parameters of the inverse denoising network based on the first loss function.

[0138] The calculation formula of the first loss function is:

[0139]

[0140] Among them, η1 represents the first loss coefficient, represents the amount of data in the training set, i represents the i-th target training data, n i Represents the number of times the i-th target training data needs to be sampled, Indicates that the j-th noise sequence of the i-th target training data is from q φ The probability that the sampled vertex number in (forward diffusion network) is exactly k, T represents T time steps, Represents sampling from the backward diffusion network to vertex v k , Represents the reconstructed mesh M i In the jth sequence, the state at the diffusion step t+1, k represents the vertex number of the current sample, represents the initial state of the i-th target training data before noise is added, Represents the jth sequence of the i-th target training data, the subsequence formed by diffusion in the first t-1 steps.

[0141] S45. Calculate a second loss function of the forward diffusion network based on the completely shielded grid and the first loss function, and adjust model parameters of the forward diffusion network based on the second loss function.

[0142] The calculation formula of the second loss function is:

[0143]

[0144] Where η2 represents the second loss coefficient; x represents the total number of noise-added sequences; represents the amount of data in the training set; i represents the i-th target training data; j represents the j-th sampling and noise sequence; Δθ represents the first loss function; σ jRepresents an indefinite length sequence of all given vertices sampled at the jth time, ending with the vertex pointed to by the end marker vertex variable; represents the initial state of the i-th target training data before noise is added, The output result is the fully shielded grid.

[0145] It should also be noted that this application can not only be used for 3D shape generation, but can also be extended to more complex 3D modeling application scenarios, such as urban modeling application scenarios, virtual reality construction application scenarios, and autonomous driving application scenarios.

[0146] To verify the effectiveness of the mesh diffusion probability model (MeshDG), this application uses multiple popular data subsets of ShapeNet (ShapeNet is a widely used data set for three-dimensional shape understanding and analysis) for experiments. For each data subset, the present invention is divided into a training set, a test set, and a validation set, where the training set accounts for 80%, the test set accounts for 15%, and the validation set accounts for 5%. The validation set is used to detect the performance and effect of the current round of the model during training. The test set is used to evaluate the performance indicators of the model in quantitative analysis experiments and compare with other models.

[0147] During the verification process, the experiment of this application used the Adam (Adaptive Moment Estimation) optimizer with a gradient clipping norm of 1.0. The learning rate was dynamically adjusted using the cosine annealing algorithm (CosineAnnealingLR), and the initial maximum learning rate was 0.0005. No linear warm-up was used during the training process. For each data subset, 5000 epochs (when a complete data set passes through the neural network once and returns once, this process is called an epoch) of training will be performed, and an early stop mechanism will be introduced. If the loss function does not decrease significantly for 20 consecutive epochs on the validation set, the training process will stop, and the best performing round of model parameters will be selected for calculation of the test set to obtain the performance indicators of the model.

[0148] In addition, in order to deal with the overfitting problem in verification, this application adopts the Dropout strategy during the training process to increase the generalization ability of the model. The core idea of ​​Dropout is to randomly discard some neurons with a certain probability (0.2 in this article) in each training round, that is, to set their output values ​​to zero. Through the Dropout strategy, the network is no longer overly dependent on certain specific neurons, but instead encourages each neuron to process the input independently. This strategy of randomly discarding neurons helps to reduce the complexity of the neural network and improve the robustness and generalization ability of the model. It should be noted that Dropout is only used during training and is disabled during testing and evaluation. Because in the test set, the output results of the model are required to be stable and reliable, without being disturbed by the random discarding of neurons.

[0149] For the polygonal mesh generated by this application, the Blender software and the off addon plug-in are used for rendering. The visualization results of the rendered polygonal mesh can be referred to Figure 5 As shown. For each polygonal mesh, place it at the origin of the coordinate system, set the resolution to 256×256, use an area light located 1.5m above the origin, set the intensity of the light source to 20W, and cover a rectangular area of ​​2.5m. At the same time, randomly and independently select 10 point light sources, and the x and y coordinates of each point light source are uniformly sampled in the interval [-2,-0.75]∪[0.75,2], and the z coordinate is uniformly sampled in the interval [0.75,2]. Finally, the camera is randomly placed 1.5m from the origin, and the rotation angle is uniformly sampled every 30 degrees between 0 and 360 degrees, thereby generating a total of 12 multi-view images to observe the quality of the polygonal mesh generated by this model from different perspectives.

[0150] like Figure 6 FIG. 1 is a block diagram of a system for generating a polygonal mesh on a point cloud provided by an embodiment of the present application. Figure 6 In the embodiment, the system 500 for generating a polygonal mesh on a point cloud comprises:

[0151] A point set acquisition module 501 is used to acquire a point set of a target product, wherein the point set includes a plurality of vertices;

[0152] A first mesh generation module 502 is used to input the point set into a mesh generation algorithm, generate a corresponding first edge set for each vertex in the point set according to the mesh generation algorithm, and generate a first mesh to be formed according to a first face set composed of all the first edge sets, wherein the length of the first edge set corresponding to each vertex varies according to the number of other vertices associated with the vertex;

[0153] The polygon mesh forming module 503 is used to obtain a polygon mesh based on the first mesh to be formed, and the polygon mesh is used to present a three-dimensional shape structure.

[0154] The embodiment of the present application also provides a terminal device. The principle block diagram of the terminal device can be as follows: Figure 7 As shown. The terminal device includes a processor, a memory, a network interface, a display screen and a temperature sensor connected via a system bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for generating a polygonal mesh on a point cloud is implemented. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor is pre-set inside the terminal device to detect the operating temperature of the internal device.

[0155] Those skilled in the art will understand that Figure 7 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present invention, and does not constitute a limitation on the terminal device to which the scheme of the present invention is applied. The specific terminal device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0156] In some embodiments, an embodiment of the present application provides a terminal device, including a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the method for generating a polygonal mesh on a point cloud provided in an embodiment of the present application.

[0157] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided by the present invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0158] The technical features of the above embodiments can be combined without changing the basic principles of the present invention. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0159] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of patent protection of the present application shall be subject to the attached claims.

Claims

1. Methods for generating polygonal meshes on point clouds, It is characterized in that include: Acquire a point set of a target product, wherein the point set includes a plurality of vertices; Inputting the point set into a mesh generation algorithm, generating a corresponding first edge set for each vertex in the point set according to the mesh generation algorithm, and generating a first to-be-formed mesh according to a first face set formed by all the first edge sets, wherein the length of the first edge set corresponding to each vertex varies according to the number of other vertices associated with the vertex; A polygonal mesh is obtained based on the first mesh to be formed, and the polygonal mesh is used to present a three-dimensional shape structure.

2. The method for generating a polygonal mesh on a point cloud according to claim 1, It is characterized in that The step of obtaining a point set of a target product, wherein the point set includes a plurality of vertices, comprises: The target product is processed based on a point cloud generation model to generate the point set, wherein the point cloud generation model is a model of a three-dimensional point set that captures and records point data on the surface of the target product through a sensor device to generate a three-dimensional point set that expresses the surface shape and characteristics of the target product.

3. The method for generating a polygonal mesh on a point cloud according to claim 1, It is characterized in that The step of inputting the point set into a mesh generation algorithm, generating a corresponding first edge set for each vertex in the point set according to the mesh generation algorithm, and generating a first mesh to be formed according to a first face set composed of all the first edge sets, wherein the length of the first edge set corresponding to each vertex varies according to the number of other vertices associated with the vertex, comprises: Setting a current vertex variable and a preset vertex variable, wherein the current vertex variable is used to point to one of the vertices in the point set, and the preset vertex variable is initialized to point to the same vertex as the current vertex variable; Set the last sampling vertex variable, the starting vertex variable, the target sampling vertex variable, the second face set, the second edge set, the end mark vertex variable and the start mark vertex variable, wherein the last sampling vertex variable, the start vertex variable and the target sampling vertex variable are all initialized to NULL, the last sampling vertex variable, the start vertex variable and the target sampling vertex variable are all used to point to one of the vertices in the point set, and the vertex pointed to by the target sampling vertex variable is not the vertex pointed to by the current vertex variable; the second face set and the second edge set are both initialized to empty sets; the end mark vertex variable is used to point to the vertex indicating the end of the prediction of all faces; the start mark vertex variable is used to point to the vertex indicating the end of the prediction of one face and the beginning of the prediction of the next face; the vertices pointed to by the end mark vertex variable and the start mark vertex variable belong to the point set; forming a second mesh to be formed based on the point set and the second surface set; The loop process is executed until a preset condition is met; the loop process includes: Extracting a sub-mesh from the second mesh to be formed, wherein the sub-mesh includes the vertex pointed to by the current vertex variable; modifying the target sampling vertex variables based on the sub-meshes and a mesh diffusion probability model of the mesh generation algorithm; In the case that the vertex pointed to by the target sampling vertex variable and the vertex pointed to by the previous sampling vertex variable are not the vertex pointed to by the start mark vertex variable, generating edges corresponding to the target sampling vertex variable and the previous sampling vertex variable, and adding them to the second edge set; When the vertex pointed to by the target sampling vertex variable is not the vertex pointed to by the start mark vertex variable, and the vertex pointed to by the previous sampling vertex variable is the vertex pointed to by the start mark vertex variable, replace the vertex pointed to by the start vertex variable with the vertex pointed to by the target sampling vertex variable; In a case where the vertex pointed to by the target sampling vertex variable is the vertex pointed to by the starting marking vertex variable, generating a third mesh to be formed based on the point set, the previous sampling vertex variable, the starting vertex variable, the second edge set and the second face set; Replace the vertex pointed to by the previous sampling vertex variable with the vertex pointed to by the target sampling vertex variable; Replace the vertex pointed to by the current vertex variable with the vertex pointed to by the target sampling vertex variable; The preset condition is that the vertex pointed to by the current vertex variable is the vertex pointed to by the end mark vertex variable.

4. The method for generating a polygonal mesh on a point cloud according to claim 3, It is characterized in that The step of modifying the target sampling vertex variable based on the sub-grid and the mesh diffusion probability model of the mesh generation algorithm comprises: Inputting the sub-grid into the grid diffusion probability model to obtain a reconstructed sub-grid; Removing vertices and edges shared with the sub-mesh from the reconstructed sub-mesh to obtain a target vertex; Replace the vertex pointed to by the target sampling vertex variable with the target vertex.

5. The method for generating a polygonal mesh on a point cloud according to claim 4, It is characterized in that The reconstructed sub-mesh has one more vertex and one more edge than the sub-mesh in structure.

6. The method for generating a polygonal mesh on a point cloud according to claim 3, It is characterized in that The step of generating a third mesh to be formed based on the point set, the last sampled vertex variable, the starting vertex variable, the second edge set and the second face set comprises: Based on the last sampled vertex variable and the starting vertex variable, generate edges corresponding to the last sampled vertex variable and the starting vertex variable, and add the second edge set to obtain the first edge set corresponding to the vertex pointed to by the preset vertex variable; Adding the first edge set to the second face set to obtain a third face set; Assign the second edge set to an empty set; Replace the vertex pointed to by the preset vertex variable with the vertex pointed to by the target sampling vertex variable; Constructing the third to-be-formed mesh based on the point set and the third face set; Replacing the value corresponding to the second grid to be formed with the value corresponding to the third grid to be formed; In a case where the vertex pointed to by the target sampling vertex variable is the vertex pointed to by the end marker vertex variable, the third face set is used as the first face set, and the third mesh to be formed is used as the first mesh to be formed.

7. The method for generating a polygonal mesh on a point cloud according to claim 3, It is characterized in that The training method of the grid diffusion probability model includes: Acquire a training set, wherein each training data in the training set is a sub-grid; Inputting target training data into the three-dimensional graph attention network and the forward diffusion network in the grid diffusion probability model to obtain a completely shielded grid, wherein the target training data is any training data in the training set; Inputting the completely shielded mesh into the three-dimensional graph attention network and the reverse denoising network in the mesh diffusion probability model to obtain a reconstructed mesh; Calculating a first loss function of the inverse denoising network based on the reconstructed grid, and adjusting model parameters of the inverse denoising network based on the first loss function; A second loss function of the forward diffusion network is calculated based on the fully shielded grid and the first loss function, and model parameters of the forward diffusion network are adjusted based on the second loss function.

8. A system for generating polygonal meshes on point clouds, It is characterized in that include: A point set acquisition module, used to acquire a point set of a target product, wherein the point set includes a plurality of vertices; A first mesh generation module to be formed is used to input the point set into a mesh generation algorithm, generate a corresponding first edge set for each vertex in the point set according to the mesh generation algorithm, and generate a first mesh to be formed according to a first face set composed of all the first edge sets, wherein the length of the first edge set corresponding to each vertex varies according to the number of other vertices associated with the vertex; A polygonal mesh forming module is used to obtain a polygonal mesh based on the first mesh to be formed, and the polygonal mesh is used to present a three-dimensional shape structure.

9. Terminal equipment, It is characterized in that include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the steps of the method for generating a polygonal mesh on a point cloud as described in any one of claims 1 to 7.

10. Computer readable storage medium, It is characterized in that Used to store a computer program, wherein the computer program enables a computer to execute the steps of the method for generating a polygonal mesh on a point cloud as described in any one of claims 1 to 7.

Citation Information

Patent Citations

  • Method for meshing point cloud data

    CN102306397A