Three-dimensional grid model generation method

Abstract

The embodiment of the invention provides a three-dimensional grid model generation method, and relates to the technical field of generative artificial intelligence, and the method comprises the steps: displaying a first three-dimensional grid model in a grid display region of an application interface, and displaying a first voxel model in a voxel operation region of the application interface; the first voxel model represents a plurality of corresponding voxel units in the first three-dimensional mesh model; in response to an editing operation on the first voxel model in the voxel operation area, generating and displaying a second voxel model; displaying a second three-dimensional grid model associated with the second voxel model in the grid display area; the second three-dimensional mesh model is obtained by updating the first three-dimensional mesh model based on the difference between the first voxel model and the second voxel model. The threshold of direct operation on the three-dimensional grid model is reduced, the three-dimensional grid model can be modified by editing the voxel model without strong professional knowledge, and the user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of generative artificial intelligence technology, and in particular to a method for generating three-dimensional mesh models. Background Technology

[0002] Editing 3D models typically requires professional 3D modeling software such as Blender, Maya, and 3ds Max. These software programs are powerful but complex to operate, requiring users to have a deep understanding of geometry, topology, and software operation skills. For ordinary users, it is difficult to directly use these software programs to edit 3D models.

[0003] To reduce the difficulty of editing 3D models, some technical solutions currently offer simple editing interfaces, but they have the following problems: they are limited to creating new voxel models and cannot import an existing, detailed mesh model for editing; the simplified editing interface still requires users to have mesh editing knowledge, and the operation is still relatively complex; and they cannot make detailed adjustments to the 3D model. Summary of the Invention

[0004] This application provides a method for generating a three-dimensional mesh model, which can efficiently and quickly edit the three-dimensional mesh model and reduce the threshold for operating the three-dimensional mesh model.

[0005] In a first aspect, embodiments of this application provide a method for generating a three-dimensional mesh model, including: The first three-dimensional mesh model is displayed in the mesh display area of ​​the application interface and the first voxel model is displayed in the voxel operation area of ​​the application interface; the first voxel model represents the corresponding multiple voxel units in the first three-dimensional mesh model; In response to an editing operation on the first voxel model in the voxel operation area, a second voxel model is generated and displayed; A second three-dimensional mesh model associated with the second voxel model is displayed in the mesh display area; the second three-dimensional mesh model is obtained by updating the first three-dimensional mesh model based on the differences between the first voxel model and the second voxel model.

[0006] In this embodiment, an application interface that links the 3D mesh display area and the voxel operation area in real time allows users to view the rendering effect of the mesh model in real time while manipulating the voxel model, thus improving the user experience. By manipulating the voxel model of the 3D mesh model, the high barrier to entry for direct manipulation of the 3D mesh model is reduced. Modifications to the 3D mesh model can be achieved by editing the voxel model without requiring strong professional knowledge, simplifying the complexity of 3D mesh model editing operations. Real-time bidirectional mapping between the 3D mesh model and the voxel model further enhances the user experience, making the linked display of the 3D mesh model and the voxel model more intuitive.

[0007] Optionally, displaying the first voxel model in the voxel operation area of ​​the application interface includes: Based on the first three-dimensional mesh model, determine the initial voxel model; Obtain image features of the first 3D mesh model from different perspectives; The image features from different perspectives are mapped to the initial voxel model to obtain the first voxel model, which is then displayed in the voxel operation area of ​​the application interface.

[0008] In this embodiment, an initial voxel model is constructed, and the image features of the three-dimensional mesh model are mapped to the initial voxel model to obtain the voxel model. This makes the voxel model have the characteristics of the three-dimensional mesh model, so that when the three-dimensional mesh model is edited based on the voxel model, it is more in line with the characteristics of the three-dimensional mesh model.

[0009] Optionally, determining the initial voxel model based on the first three-dimensional mesh model includes: The three-dimensional mesh model is then watertightened. The three-dimensional space containing the watertightened three-dimensional mesh model is divided into a uniform voxel space; The initial voxel model is constructed based on the location occupied by the watertightened 3D mesh model in the voxel space.

[0010] In this embodiment, by performing watertight processing on the 3D mesh model, errors are avoided during subsequent editing operations on the voxel model. The initial voxel model is determined based on the location occupied by the 3D mesh model in voxel space, ensuring that the initial voxel model perfectly matches the location occupied by the 3D mesh model.

[0011] Optionally, obtaining the image features of the first 3D mesh model from different viewpoints includes: Multiple two-dimensional images of the first three-dimensional mesh model were acquired from different perspectives; Obtain the two-dimensional feature vector of the two-dimensional image from each viewpoint; The image features of the first three-dimensional mesh model under different viewpoints are determined based on the two-dimensional feature vectors under each viewpoint.

[0012] In this embodiment, by acquiring two-dimensional images of the three-dimensional mesh model from different perspectives, and obtaining two-dimensional feature vectors from different perspectives, the image features of the three-dimensional mesh model from different perspectives are determined based on the two-dimensional feature vectors from each perspective. This ensures the accuracy of determining the image features of the three-dimensional mesh model. By acquiring two-dimensional feature vectors from multiple perspectives, deviations are avoided when determining the image features of the three-dimensional mesh model.

[0013] Optionally, the initial voxel model is composed of multiple voxel units; each voxel unit in the initial voxel model has three-dimensional coordinate information and occupancy status information. The step of mapping the image features from different viewpoints to the initial voxel model to obtain the first voxel model includes: Determine the voxel unit corresponding to the image features under any viewpoint in the voxel model; For any voxel unit in the voxel model, multiple image features projected onto the voxel unit are processed to obtain a multidimensional feature representation of the voxel unit, thereby obtaining a first voxel model; any voxel unit in the first voxel model has three-dimensional coordinate information, occupancy status information, and multidimensional feature representation.

[0014] In this embodiment, image features from different perspectives are projected onto corresponding voxel units to obtain multiple image features for each voxel unit. These multiple image features are then processed to obtain the image features for each voxel unit, thus obtaining a voxel model. The voxel model is determined by the image features of the voxel units, so that the voxel model can better reflect the three-dimensional mesh model.

[0015] Optionally, an editing component is provided in the voxel operation area; the response to the editing operation on the first voxel model in the voxel operation area includes: The editing type is determined based on the user's actions on the editing component; Collect the voxel units corresponding to the user's editing operations on the first voxel model; Record the editing type, the voxel unit corresponding to the editing operation, and the position information of the voxel unit.

[0016] In this embodiment, by recording the editing type, the voxel unit corresponding to the editing operation, and the position of the voxel unit, the specific editing position and editing content can be determined, which facilitates subsequent comparison of differences before and after editing.

[0017] Optionally, the second three-dimensional mesh model is obtained by updating the first three-dimensional mesh model based on the differences between the first voxel model and the second voxel model, including: The first voxel model and the second voxel model are respectively input into the encoder to obtain the first high-dimensional voxel features and the second high-dimensional voxel features; The first high-dimensional voxel feature and the second high-dimensional voxel feature are input into the three-dimensional generation model to obtain the differential mesh feature; The difference grid features are input into the decoder to obtain the difference grid; The first three-dimensional mesh model is updated based on the difference mesh to obtain the second three-dimensional mesh model.

[0018] In this embodiment, high-dimensional feature vectors before and after editing are obtained by inputting the voxel models into the encoder. Comparing these high-dimensional feature vectors allows for a more accurate determination of the differences before and after editing. The high-dimensional feature vectors are then input into the generation model to determine the difference mesh, and the first 3D mesh model is updated based on this difference mesh to make the update of the second 3D mesh model more precise.

[0019] Optionally, the encoder and the decoder are jointly trained; wherein, during the training process, a first supervision signal and a second supervision signal are set, the first supervision signal being used to determine the accuracy of the geometry of the 3D mesh model, and the second supervision signal being used to determine the accuracy of the mesh texture of the 3D mesh model.

[0020] In this embodiment, by jointly training the encoder and decoder, the encoding and decoding process becomes more consistent, avoiding inconsistencies caused by the separation of encoding and decoding, which would prevent the output of an accurate 3D mesh model.

[0021] Optionally, the encoder includes alternately stacked sparse convolutional layers and attention layers; the sparse convolutional layers are used to extract the geometry of the voxel model, and the attention layers are used to determine the overall structure of the voxel model. The decoder is used to predict the vertex position offset of each voxel and the connection probability of the face, and to generate the difference mesh based on the vertex position offset of each voxel and the connection probability of the face.

[0022] In this embodiment of the application, by setting encoders and decoders with different structures, the encoding and decoding process is made to better conform to the encoding and decoding of voxels and meshes in this embodiment of the application, thereby obtaining more accurate voxel models and mesh models.

[0023] Optionally, updating the first 3D mesh model based on the difference mesh to obtain the second 3D mesh model includes: If the editing operation is an add operation, then based on the position information of the voxel unit corresponding to the editing operation, the difference mesh is added to the first 3D mesh model to obtain the second 3D mesh model, or If the editing operation is a deletion operation, then based on the position information of the voxel unit corresponding to the editing operation, the difference mesh is deleted from the first 3D mesh model to obtain the second 3D mesh model. If the editing operation is a modification operation, then the difference mesh is mapped to the first three-dimensional mesh model according to the position information of the voxel unit corresponding to the editing operation to obtain the second three-dimensional mesh model.

[0024] In this embodiment, different editing operation types are used to edit voxel units, making the editing operation from voxel model to mesh model more diversified. It is not limited to simply adding or deleting to modify the 3D mesh model, making this solution more flexible.

[0025] Secondly, embodiments of this application provide a three-dimensional mesh model generation apparatus, comprising: The display module is used to display a first three-dimensional mesh model in the mesh display area of ​​the application interface and a first voxel model in the voxel operation area of ​​the application interface; the first voxel model represents a plurality of voxel units corresponding to the first three-dimensional mesh model. A response module is configured to generate and display a second voxel model in response to an editing operation on the first voxel model in the voxel operation area. An update module is used to display a second three-dimensional mesh model associated with the second voxel model in the mesh display area; the second three-dimensional mesh model is obtained by updating the first three-dimensional mesh model based on the differences between the first voxel model and the second voxel model.

[0026] Optionally, the display module is specifically used for: Based on the first three-dimensional mesh model, determine the initial voxel model; Obtain image features of the first 3D mesh model from different perspectives; The image features from different perspectives are mapped to the initial voxel model to obtain the first voxel model, which is then displayed in the voxel operation area of ​​the application interface.

[0027] Optionally, the display module is specifically used for: The three-dimensional mesh model is then watertightened. The three-dimensional space containing the watertightened three-dimensional mesh model is divided into a uniform voxel space; The initial voxel model is constructed based on the location occupied by the watertightened 3D mesh model in the voxel space.

[0028] Optionally, the display module is specifically used for: Multiple two-dimensional images of the first three-dimensional mesh model were acquired from different perspectives; Obtain the two-dimensional feature vector of the two-dimensional image from each viewpoint; The image features of the first three-dimensional mesh model under different viewpoints are determined based on the two-dimensional feature vectors under each viewpoint.

[0029] Optionally, the initial voxel model is composed of multiple voxel units; each voxel unit in the initial voxel model has three-dimensional coordinate information and occupancy status information. Optionally, the display module is specifically used for: Determine the voxel unit corresponding to the image features under any viewpoint in the voxel model; For any voxel unit in the voxel model, multiple image features projected onto the voxel unit are processed to obtain a multidimensional feature representation of the voxel unit, thereby obtaining a first voxel model; any voxel unit in the first voxel model has three-dimensional coordinate information, occupancy status information, and multidimensional feature representation.

[0030] Optionally, an editing component is provided in the voxel operation area; Optionally, the response module is specifically used for: The response to the editing operation on the first voxel model in the voxel operation area includes: The editing type is determined based on the user's actions on the editing component; Collect the voxel units corresponding to the user's editing operations on the first voxel model; Record the editing type, the voxel unit corresponding to the editing operation, and the position information of the voxel unit.

[0031] Optionally, the update module is specifically used for: The first voxel model and the second voxel model are respectively input into the encoder to obtain the first high-dimensional voxel features and the second high-dimensional voxel features; The first high-dimensional voxel feature and the second high-dimensional voxel feature are input into the three-dimensional generation model to obtain the differential mesh feature; The difference grid features are input into the decoder to obtain the difference grid; The first three-dimensional mesh model is updated based on the difference mesh to obtain the second three-dimensional mesh model.

[0032] Optionally, the encoder and the decoder are jointly trained; wherein, during the training process, a first supervision signal and a second supervision signal are set, the first supervision signal being used to determine the accuracy of the geometry of the 3D mesh model, and the second supervision signal being used to determine the accuracy of the mesh texture of the 3D mesh model.

[0033] Optionally, the encoder includes alternately stacked sparse convolutional layers and attention layers; the sparse convolutional layers are used to extract the geometry of the voxel model, and the attention layers are used to determine the overall structure of the voxel model. The decoder is used to predict the vertex position offset of each voxel and the connection probability of the face, and to generate the difference mesh based on the vertex position offset of each voxel and the connection probability of the face.

[0034] Optionally, the update module is specifically used for: If the editing operation is an add operation, then based on the position information of the voxel unit corresponding to the editing operation, the difference mesh is added to the first 3D mesh model to obtain the second 3D mesh model, or If the editing operation is a deletion operation, then based on the position information of the voxel unit corresponding to the editing operation, the difference mesh is deleted from the first 3D mesh model to obtain the second 3D mesh model. If the editing operation is a modification operation, then the difference mesh is mapped to the first three-dimensional mesh model according to the position information of the voxel unit corresponding to the editing operation to obtain the second three-dimensional mesh model.

[0035] Thirdly, embodiments of this application provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the methods described above.

[0036] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of any of the methods described above.

[0037] Fifthly, this application provides a computer program product, the computer program product including a computer program stored on a computer-readable storage medium, the computer program including program instructions, which, when executed by a computer device, cause the computer device to perform the steps of any of the methods described above.

[0038] In this embodiment, an application interface that links the 3D mesh display area and the voxel operation area in real time allows users to view the rendering effect of the mesh model in real time while manipulating the voxel model, thus improving the user experience. By manipulating the voxel model of the 3D mesh model, the high barrier to entry for direct manipulation of the 3D mesh model is reduced. Modifications to the 3D mesh model can be achieved by editing the voxel model without requiring strong professional knowledge, simplifying the complexity of 3D mesh model editing operations. Real-time bidirectional mapping between the 3D mesh model and the voxel model further enhances the user experience, making the linked display of the 3D mesh model and the voxel model more intuitive. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A system architecture diagram provided for an embodiment of this application; Figure 2 A flowchart (I) illustrating a method for generating a three-dimensional mesh model provided in this application embodiment; Figure 3a An application interface (a) provided for an embodiment of this application; Figure 3b An application interface (II) provided for an embodiment of this application; Figure 4 A flowchart illustrating how a first voxel model is displayed in the voxel operation area of ​​an application interface, as provided in an embodiment of this application. Figure 5 A schematic diagram of a process for determining an initial voxel model provided in an embodiment of this application; Figure 6 A flowchart illustrating a process for obtaining image features of a first three-dimensional mesh model from different viewpoints, provided in an embodiment of this application; Figure 7 This is a schematic diagram illustrating a process for updating a first three-dimensional mesh model, provided as an embodiment of this application. Figure 8 A flowchart (II) illustrating a method for generating a three-dimensional mesh model provided in this application embodiment; Figure 9 This is a schematic diagram of the structure of a three-dimensional mesh model generation device provided in an embodiment of this application; Figure 10This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and beneficial effects of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0042] 3D mesh model: A common method of representing 3D models, which uses a mesh structure consisting of vertices, edges and faces to define geometry, and adds texture maps to define surface color and details.

[0043] Voxel: Short for 3D pixel. It represents a cubic unit on a 3D mesh in 3D space and is the basic unit that constitutes the data of a 3D mesh model. In this scheme, each voxel typically contains 3D coordinates, multi-dimensional features, and occupancy status information (indicating whether the voxel belongs to the 3D mesh model).

[0044] To facilitate understanding of this solution, its application scenarios are described below.

[0045] Currently, 3D mesh model editing mainly relies on professional 3D modeling software (such as Blender, Maya, 3ds Max, etc.). These software programs are powerful, but complex to operate, with a steep learning curve, requiring users to have a deep understanding of geometry, topology, and software operation skills. For ordinary users without a professional background, directly editing complex 3D mesh models (including modifying geometry, adding or deleting parts, and adjusting color and texture) is a very difficult, or even impossible, task.

[0046] To lower the barrier to entry, some technical solutions attempt to provide a simpler editing interface: (1) Voxel / block-based editing tools (such as Goxel, MagicaVoxel): These tools allow users to create and edit models using voxels like building blocks, making them intuitive and easy to use. However, they are usually limited to creating new voxel models and cannot easily import an existing, detailed 3D mesh model for editing. They also lack the ability to back-map the user's operations in voxel space back to the original 3D mesh model in real time and with high precision. The user edits the voxels themselves, rather than manipulating the original 3D mesh model through voxels.

[0047] (2) Mesh Simplification and Proxy Editing: Some professional software provides a mesh simplification function to generate a low-poly (e.g., Low-Poly) proxy model for users to roughly edit, and then transfer the edited result back to the high-precision 3D mesh model. This method has significant problems: First, the simplified proxy model is usually a mesh structure (although with fewer faces), and users still need some mesh editing knowledge; Second, editing operations (such as moving vertices and extrude faces) are still not intuitive for non-professional users; Third, and most importantly, the editing mapping from low-poly to high-poly is usually non-real-time, batch-processed, and the algorithm is complex, making it difficult to guarantee the accurate transmission of editing intentions (especially when it involves adding or deleting structures), which can easily lead to distortion, breakage, or topological errors in the high-poly model.

[0048] (3) Intuitive brush-based editing: Some software provides brush tools for pushing, pulling, smoothing, sculpting and other operations on 3D mesh models. Although these tools are relatively intuitive, they are mainly for local detail adjustments on the surface of 3D mesh models (such as sculpting), and are difficult to use for precise structural modifications (such as adding an arm or deleting a part) or large-scale adjustments to the overall structure, and have limited ability to handle complex topological structures.

[0049] Based on the above technical solutions, this application provides a method for generating 3D mesh models, establishing a real-time bidirectional mapping mechanism based on voxel mediation, significantly reducing the threshold for editing 3D mesh models. Specifically, users can perform simple, easy-to-use, fast, and accurate 3D mesh editing by operating a dual-interface linkage system. The application interface provides two synchronized view windows: a mesh display area and a voxel operation area. The voxel operation area uses intuitive voxel editing tools, allowing users to perform operations such as erasing, adding, or deleting voxels; the mesh display area renders the 3D mesh model corresponding to the voxel operation area in real time. When a user performs an editing operation in the voxel operation area, the operation data is immediately captured, and the 3D mesh update is calculated through a bidirectional twin representation mapping algorithm. The mesh display area completes real-time rendering and refresh, achieving an interactive experience. The two interfaces are tightly coupled through a shared twin representation data structure, ensuring that any modification in the voxel space is accurately reflected in the mesh space. To this end, this solution constructs an automatic conversion system from mesh to voxel and a digital twin generation system, as well as a real-time bidirectional mapping mechanism from voxel editing to mesh modification.

[0050] See Figure 1 This is a system architecture diagram provided in an embodiment of the present application. The system architecture includes a terminal device 101 and a server 102.

[0051] Terminal device 101 is pre-installed with a business application for text matching, which may be a client application, a web application, a mini-program application, etc. Terminal device 101 may be a smartphone, a desktop laptop, a computer, etc., but is not limited to these.

[0052] Server 102 is the backend server for business applications. Server 102 can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0053] See Figure 2 The above is a flowchart illustrating a method for generating a three-dimensional mesh model according to an embodiment of this application, including the following steps: Step 201: Display the first three-dimensional mesh model in the mesh display area of ​​the application interface and display the first voxel model in the voxel operation area of ​​the application interface; the first voxel model represents the corresponding multiple voxel units in the first three-dimensional mesh model.

[0054] Specifically, such as Figure 3a and Figure 3b As shown, this is an application interface provided in an embodiment of this application. The display area of ​​the application interface is divided into two parts: a mesh display area on the left and a voxel operation area on the right. A three-dimensional mesh model is displayed in the mesh display area, and a voxel model is displayed in the voxel operation area. The voxel model represents multiple voxel units corresponding to the three-dimensional mesh model.

[0055] After acquiring the first 3D mesh model, it is displayed in the mesh display area and simultaneously in the voxel operation area. The first 3D mesh model is the original 3D mesh model without any editing operations, while the first voxel model is a voxel model generated based on the first 3D mesh model. The first voxel model includes multiple voxel units and is generated in real time based on the first 3D mesh model.

[0056] The application interface provided in this application embodiment is only for understanding this solution, but does not limit the specific form of the application interface.

[0057] Step 202: In response to the editing operation on the first voxel model in the voxel operation area, generate and display the second voxel model.

[0058] Specifically, voxel models can be edited in the voxel manipulation area. For example, editing the first voxel model will result in a second voxel model, which will also be displayed in the voxel manipulation area.

[0059] Step 203: Display the second three-dimensional mesh model associated with the second voxel model in the mesh display area; the second three-dimensional mesh model is obtained by updating the first three-dimensional mesh model based on the differences between the first voxel model and the second voxel model.

[0060] Specifically, after editing the first voxel model in the voxel manipulation area, a second voxel model is obtained. A second 3D mesh model associated with the second voxel model is then displayed in the mesh display area. The second 3D mesh model is the 3D mesh model obtained after the editing operation. The second 3D mesh model is obtained by updating the first 3D mesh model based on the differences between the first and second voxel models.

[0061] In some embodiments, a first voxel model is displayed in the voxel manipulation area of ​​the application interface, such as... Figure 4 As shown, it includes the following steps: Step 401: Determine the initial voxel model based on the first three-dimensional mesh model.

[0062] Step 402: Obtain the image features of the first 3D mesh model from different perspectives.

[0063] Step 403: Map the image features from different perspectives to the initial voxel model to obtain the first voxel model and display it in the voxel operation area of ​​the application interface.

[0064] Specifically, firstly, an initial voxel model is determined based on the first 3D mesh model. Then, the image features of the 3D mesh model are uniformly sampled from multiple different viewpoints. For each viewpoint, the 3D mesh is placed at the center of the viewpoint and rendered. For example, the 3D mesh is rendered as 1021... A 1024-pixel image is used to ensure that the image features of the 3D mesh model are fully captured from different angles. Finally, the image features from different viewpoints are mapped to the initial voxel model to obtain the first voxel model.

[0065] In this embodiment, an initial voxel model is constructed, and the image features of the three-dimensional mesh model are mapped to the initial voxel model to obtain the voxel model. This makes the voxel model have the characteristics of the three-dimensional mesh model, so that when the three-dimensional mesh model is edited based on the voxel model, it is more in line with the characteristics of the three-dimensional mesh model.

[0066] In some embodiments, an initial voxel model is determined based on a first three-dimensional mesh model, such as... Figure 5 As shown, it includes the following steps: Step 501: Perform watertight processing on the 3D mesh model.

[0067] Step 502: Divide the three-dimensional space of the watertightened three-dimensional mesh model into a uniform voxel space.

[0068] Step 503: Construct an initial voxel model based on the location of the watertight 3D mesh model in voxel space.

[0069] Specifically, the 3D mesh model is first watertightened to address issues such as holes and non-flowing edges. The ManifoldPlus algorithm can be used for watertightening. Then, the 3D space containing the mesh model is divided into a uniform voxel space, which can be 64 voxels. 64 It can be composed of 64 voxel units, or other voxel units. Finally, it is determined whether each voxel unit in the voxel space is surrounded by the 3D mesh model to form the initial voxel model. The specific conversion rule is as follows: if the center point of the voxel unit is located within the 3D mesh model, the voxel unit is marked as "occupied"; otherwise, it is marked as "free". Here, "1" represents "occupied" and "0" represents "unoccupied".

[0070] In this embodiment, by performing watertight processing on the 3D mesh model, errors are avoided during subsequent editing operations on the voxel model. The initial voxel model is determined based on the location occupied by the 3D mesh model in voxel space, ensuring that the initial voxel model perfectly matches the location occupied by the 3D mesh model.

[0071] In some embodiments, image features of the first 3D mesh model are obtained from different viewpoints, such as... Figure 6 As shown, it includes the following steps: Step 601: Collect multiple 2D images of the first 3D mesh model from different perspectives.

[0072] Step 602: Obtain the two-dimensional feature vector of the two-dimensional image from each viewpoint.

[0073] Step 603: Determine the image features of the first three-dimensional mesh model under different viewpoints based on the two-dimensional feature vectors under each viewpoint.

[0074] Specifically, a first 3D mesh model acquired from multiple viewpoints is input into a feature extractor, such as a pre-trained Dino-v2 feature extractor, to obtain multi-dimensional feature vectors for each viewpoint. The dimension of the acquired image features is determined based on the first 3D mesh model; for example, in this embodiment, 1536-dimensional feature vectors are obtained for each viewpoint. A voxel projection algorithm is used to back-project the feature vectors of each viewpoint onto the 3D mesh model. Since the 2D images of the 3D mesh model are acquired from different viewpoints, multiple 2D feature vectors are obtained for the same voxel unit. For each voxel unit, its corresponding pixel position in the 2D images across all viewpoints is determined. The average of all feature vectors for each voxel unit is taken to obtain the feature vector for that voxel unit. For example, voxel unit 1 is included in viewpoints 1, 2, and 3. The 2D feature vector obtained in viewpoint 1 is (a1, b1, c1), the 2D feature vector obtained in viewpoint 2 is (a2, b2, c2), and the 2D feature vector obtained in viewpoint 3 is (a3, b3, c3). Therefore, the eigenvector of voxel unit 1 is ((a1+a2+a3) / 3, (b1+b2+b3) / 3, (c1+c2+c3) / 3).

[0075] In this embodiment, by acquiring two-dimensional images of the three-dimensional mesh model from different perspectives, and obtaining two-dimensional feature vectors from different perspectives, the image features of the three-dimensional mesh model from different perspectives are determined based on the two-dimensional feature vectors from each perspective. This ensures the accuracy of determining the image features of the three-dimensional mesh model. By acquiring two-dimensional feature vectors from multiple perspectives, deviations are avoided when determining the image features of the three-dimensional mesh model.

[0076] In some embodiments, the initial voxel model is composed of multiple voxel units; each voxel unit in the initial voxel model has three-dimensional coordinate information and occupancy status information.

[0077] Specifically, each voxel unit contains three-dimensional coordinates (x, y, z) and an occupancy state (0 or 1).

[0078] In some embodiments, image features from different viewpoints are mapped to an initial voxel model to obtain a first voxel model, including: Determine the voxel unit corresponding to the image features in the voxel model under any viewpoint; for any voxel unit in the voxel model, process multiple image features projected onto the voxel unit to obtain the multidimensional feature representation of the voxel unit, thereby obtaining the first voxel model; any voxel unit in the first voxel model has three-dimensional coordinate information, occupancy state information and multidimensional feature representation.

[0079] Specifically, voxel units may exist in image features from multiple viewpoints. The voxel units in the voxel model are determined based on the image features of each viewpoint. The feature representation of each voxel unit is calculated using the method described above, such as a 1536-dimensional feature representation, resulting in the first voxel model. This first voxel model includes the three-dimensional coordinate information, occupancy status, and multi-dimensional features of all voxel units.

[0080] In this embodiment, image features from different perspectives are projected onto corresponding voxel units to obtain multiple image features for each voxel unit. These multiple image features are then processed to obtain the image features for each voxel unit, thus obtaining a voxel model. The voxel model is determined by the image features of the voxel units, so that the voxel model can better reflect the three-dimensional mesh model.

[0081] In some embodiments, an editing component is provided in the voxel operation area; in response to an editing operation on a first voxel model in the voxel operation area, the method includes: determining the editing type based on the user's operation on the editing component; collecting the voxel unit corresponding to the user's editing operation on the first voxel model; and recording the editing type, the voxel unit corresponding to the editing operation, and the position information of the voxel unit.

[0082] Specifically, such as Figure 3a and Figure 3b As shown, an editing component 301 is provided in the voxel operation area. When the user performs editing operations on the voxel model, they edit the voxel model through the editing component 301. Based on the user's operation on the editing component, the editing type, the voxel unit corresponding to the editing operation, and the position information of the edited voxel unit are determined. Among them, the editing types include: adding, deleting, modifying, etc.

[0083] In this embodiment, by recording the editing type, the voxel unit corresponding to the editing operation, and the position of the voxel unit, the specific editing position and editing content can be determined, which facilitates subsequent comparison of differences before and after editing.

[0084] In some embodiments, the second three-dimensional mesh model is obtained by updating the first three-dimensional mesh model based on the differences between the first voxel model and the second voxel model, such as... Figure 7 As shown, it includes the following steps: Step 701: Input the first voxel model and the second voxel model into the encoder respectively to obtain the first high-dimensional voxel features and the second high-dimensional voxel features.

[0085] Step 702: Input the first high-dimensional voxel features and the second high-dimensional voxel features into the 3D generation model to obtain the differential mesh features.

[0086] Step 703: Input the difference grid features into the decoder to obtain the difference grid.

[0087] Step 704: Update the first three-dimensional mesh model based on the difference mesh to obtain the second three-dimensional mesh model.

[0088] Specifically, the process of converting a voxel model into a 3D mesh model is as follows: First, the first and second voxel models are input into the encoder, which outputs first and second high-dimensional voxel features. Then, these first and second high-dimensional voxel features are input into the 3D generative model. Due to the differences between the first and second high-dimensional voxel features, the 3D generative model outputs difference mesh features. Finally, these difference mesh features are input into the decoder to obtain the difference mesh. The first 3D model is updated based on the difference mesh to obtain the second 3D mesh.

[0089] For example, such as Figure 3a and Figure 3b As shown, the first 3D mesh model, "a kitten," is displayed in the mesh display area, and the corresponding voxel model of "a kitten" is displayed in the voxel operation area. When the user edits "a kitten" in the voxel operation area, "a backpack" is added to the back of "a kitten," forming a voxel model of "a kitten carrying a backpack."

[0090] When generating a 3D mesh model after modifying the voxel model, firstly, voxel models of "a kitten" and "a kitten carrying a backpack" are obtained, and then input into the encoder to obtain the first high-dimensional voxel features of "a kitten" and the second high-dimensional voxel features of "a kitten carrying a backpack." The first and second high-dimensional voxel features of "a kitten" and "a kitten carrying a backpack" are then input into the 3D generated model to obtain the difference mesh features between "a kitten" and "a kitten carrying a backpack," i.e., the mesh features of "a backpack." Finally, the mesh features of "a backpack" are input into the decoder to obtain the difference mesh "a backpack." Based on the difference mesh "a backpack," the first 3D mesh model "a kitten" is updated to obtain the second 3D mesh model "a kitten carrying a backpack."

[0091] In this embodiment, high-dimensional feature vectors before and after editing are obtained by inputting the voxel models into the encoder. Comparing these high-dimensional feature vectors allows for a more accurate determination of the differences before and after editing. The high-dimensional feature vectors are then input into the generation model to determine the difference mesh, and the first 3D mesh model is updated based on this difference mesh to make the update of the second 3D mesh model more precise.

[0092] In some embodiments, the encoder and decoder are jointly trained; wherein, during the training process, a first supervision signal and a second supervision signal are provided, the first supervision signal being used to determine the accuracy of the geometry of the 3D mesh model, and the second supervision signal being used to determine the accuracy of the mesh texture of the 3D mesh model.

[0093] Specifically, during training, the encoder and decoder are trained jointly, using two supervision signals: a first supervision signal and a second supervision signal. The first supervision signal is used to determine the geometric loss of the 3D mesh model, calculate the symbolic spatial distance field (SDF) loss between the generated mesh model and the target mesh model, and measure the accuracy of the mesh geometry. As shown in Equation (1), the first supervision signal is:

[0094] in, and These are the predicted and true symbolic distance values, respectively, and N is the number of points uniformly sampled in 3D space (usually 50,000 points).

[0095] The second supervision signal is used to determine the texture loss of the 3D mesh model. It calculates the MSE loss of the rendered images of the generated mesh model and the target mesh model under random viewpoints to ensure the accuracy of the texture of the 3D mesh model. As shown in formula (2), it is the second supervision signal:

[0096] Where W, H, and C represent the width, height, and number of channels of the rendered image, respectively (usually...). , ), and Predicted and real images at location The pixel value of the c-th channel.

[0097] The training data comes from 3D model datasets such as Objaverse, containing 500,000 3D models of different categories. Training is conducted in two phases: the first phase uses randomly initialized parameters and employs the AdamW optimizer with a learning rate of 0.0001 for 200 epochs. This scheme innovatively uses a variational autoencoder (VAE) to transform voxel representations into raw 3D mesh data. The VAE consists of an encoder and a decoder. The encoder receives the position coordinates and feature data of the first and second voxel models, compressing them into high-dimensional feature vectors. The decoder receives the high-dimensional feature vectors and recovers a high-resolution 3D symbolic distance (SDF) field and a texture mapping field. Spatial sampling is used in these two 3D fields to obtain the texture mesh representation. Both the encoder and decoder innovatively combine sparse convolutional layers and attention layers. The sparse convolutional layers excel at extracting local structural features, while the attention layers excel at modeling long-range dependencies. This VAE is trained using a large-scale 3D dataset and employs spatial distance field loss and rendering loss as supervision signals.

[0098] In this embodiment, by jointly training the encoder and decoder, the encoding and decoding process becomes more consistent, avoiding inconsistencies before and after encoding and decoding caused by the separation of encoding and decoding, which would prevent the output of an accurate 3D mesh model.

[0099] In some embodiments, the encoder includes alternately stacked sparse convolutional layers and attention layers; the sparse convolutional layers are used to extract the geometry of the voxel model, and the attention layers are used to determine the overall structure of the voxel model. The decoder is used to predict the vertex position offset of each voxel and the connection probability of the face, and generates a difference mesh based on the vertex position offset of each voxel and the connection probability of the face.

[0100] Specifically, the encoder consists of multiple layers of sparse convolutional layers and multiple layers of attention layers stacked alternately, such as 3 layers of sparse convolutional layers and 3 layers of attention layers. Taking a 64×64×64 voxel model as an example, local geometric features are first extracted through sparse convolutional layers (kernel size 3×3×3, stride 2), outputting a 16×16×16 sparse feature map; then, global dependencies are calculated through attention layers to enhance the understanding of the overall structure of the voxel model; this process is repeated, finally outputting an 8×8×8×512 compressed feature vector.

[0101] The decoder employs a symmetrical structure: first, it progressively upsamples the feature map through three transposed sparse convolutional layers (kernel size 3×3×3, stride 2), with each layer followed by an attention layer to optimize details; after upsampling to a resolution of 64×64×64, a fully connected layer predicts the offset of each grid vertex position and the facet connectivity. Specifically, the decoder output contains two branches: one branch predicts the vertex position offset (3D vector) of each voxel in the 64×64×64 voxels, and the other branch predicts the facet connectivity probability (6D vector, corresponding to the connection possibilities of the 6 faces of the cube); based on these predictions, the Marching Cubes algorithm is used to generate the final 3D mesh model.

[0102] In this embodiment of the application, by setting encoders and decoders with different structures, the encoding and decoding process is made to better conform to the encoding and decoding of voxels and meshes in this embodiment of the application, thereby obtaining more accurate voxel models and mesh models.

[0103] The 3D generative model is trained in the following way: After the user performs editing operations on the first and second voxel models in the voxel operation area, the voxel models before and after editing are input into the encoder to obtain the first and second high-dimensional voxel features before and after editing, as shown in formula (3):

[0104] in, and These represent the first voxel model and the second voxel model before editing, respectively. and These represent the first and second high-dimensional voxel features after editing, respectively. For feature dimension, The encoder function is defined as follows: The first high-dimensional voxel features and the second high-dimensional voxel features are concatenated together and injected into the generator network module as a conditional vector, as shown in formula (4):

[0105] This vector encodes the semantic information of the user's editing operation, providing guidance for the generative network module. The generative network module uses DiT (Diffusion Transformer) as the basic network structure, mapping the Gaussian distribution to the aforementioned feature vector distribution, and learning the mapping from noise to feature vectors through the following diffusion process, as shown in Equation (5):

[0106] in, For the target feature vector, Features after adding noise For noise scheduling parameters, The noise is Gaussian. During training, the model predicts noise as follows: .

[0107] The loss function is defined as the mean square error between the predicted noise and the actual noise, as shown in formula (6):

[0108] in, Indicates model parameters, This is the diffusion time step.

[0109] When using a generative network model, the DiT network generates models that conform to random noise step by step through the following back-diffusion process. The guided feature vector is shown in formula (7):

[0110] in, , The standard deviation of noise. .

[0111] The final generated feature vector The input decoder generates a smooth, topologically sound 3D mesh that accurately reflects the content edited in the voxel interface, as shown in formula (8):

[0112] in, For decoder functions, This is the output 3D mesh.

[0113] The DiT model training process consists of two stages: a pre-training stage and a fine-tuning stage. The pre-training stage uses 500,000 3D models to pre-train the DiT network on a standard image-to-3D generation task, learning the distribution characteristics of the 3D models. The fine-tuning stage constructs a dataset containing 40,000 triplet samples. The comparison training dataset, in which This is the target mesh obtained using a standard mesh generation algorithm. During fine-tuning, the voxel differences before and after editing are considered. As a conditional signal, use diffusion loss Supervised training learns the mapping relationship between editing operations and mesh changes. The fine-tuning process optimizes the following objective function, as shown in Equation (9):

[0114] in, These are the target network parameters to be optimized.

[0115] In some embodiments, updating the first three-dimensional mesh model based on the difference mesh to obtain the second three-dimensional mesh model includes: If the editing operation is an add operation, then based on the position information of the voxel unit corresponding to the editing operation, a difference mesh is added to the first 3D mesh model to obtain the second 3D mesh model, or If the edit operation is a deletion operation, then based on the position information of the voxel unit corresponding to the edit operation, the difference mesh is deleted from the first 3D mesh model to obtain the second 3D mesh model. If the editing operation is a modification operation, then based on the position information of the voxel unit corresponding to the editing operation, the difference mesh is mapped to the first three-dimensional mesh model to obtain the second three-dimensional mesh model.

[0116] Specifically, for different types of editing operations, the editing operation is performed based on the location of the voxel unit corresponding to the editing operation.

[0117] If the editing operation is an add operation, then a difference mesh is added at the position of the corresponding voxel unit to obtain the modified 3D mesh model; If the editing operation is a deletion operation, the difference mesh will be deleted at the corresponding voxel unit position to obtain the modified 3D mesh model; If the editing operation is a modification operation, then the difference mesh is modified at the corresponding voxel unit position to obtain the modified 3D mesh model.

[0118] For example, such as Figure 3a and Figure 3b As shown, the editing operation is to add "a backpack" to the back of "a kitten". After adding the differential mesh "a backpack" to the corresponding voxel unit, the modified 3D mesh model of "a kitten carrying a backpack" is obtained in the mesh display area.

[0119] In this embodiment, different editing operation types are used to edit voxel units, making the editing operation from voxel model to mesh model more diversified. It is not limited to simply adding or deleting to modify the 3D mesh model, making this solution more flexible.

[0120] To facilitate understanding of this solution, the following example illustrates the implementation of this application using a user-edited 3D model of a kitten carrying a backpack. Figure 8 As shown, it includes the following steps.

[0121] Step 801: Obtain the 3D mesh model file of "a kitten" input by the user.

[0122] This file contains vertex, edge, face, and texture map data, and automatically initiates the mesh-to-voxel conversion process, converting the first 3D mesh model of "a kitten" into the first voxel model.

[0123] Step 802: Uniformly sample 120 rendering perspectives in a 3D spherical coordinate system, with each perspective placing the 3D mesh model at the center, and render to generate an RGB image.

[0124] The rendering perspective covers all key directions, capturing the overall shape and color distribution.

[0125] Step 803: Input the rendered image into the neural feature extractor to extract image features and perform watertight processing.

[0126] The first 3D mesh model is transformed into a watertight mesh using a watertightening algorithm, and then voxelized to generate a 64-dimensional mesh. 64 64-voxel representation. Each voxel block contains spatial coordinate information, and 120 image features are back-projected onto the voxel and averaged to obtain the feature data of each voxel block, forming a coarse-grained digital twin.

[0127] Step 804: Edit the voxel model of "a kitten".

[0128] A mesh-voxel linkage application interface is provided to users, allowing them to edit the first voxel model of a "cat". Specifically, users select a position behind the "cat" in the voxel operation area and perform an "add voxel" operation, placing multiple voxel units to simulate the shape of a backpack. This editing operation is captured in real time, recording the editing type (addition), position, and the modified set of voxel units.

[0129] Step 805: Based on the user's editing operations, input the voxel models before and after editing into the encoder to obtain high-dimensional feature vectors.

[0130] The algorithm analyzes user editing operations, recording the location of the user's addition and the shape of the voxel unit. The original voxel representation before editing and the voxel representation of the backpack after editing are then input into the encoder part of a variational autoencoder (VAE). The encoder combines sparse convolutional layers and attention layers to compress the two representations into high-dimensional feature vectors before and after editing.

[0131] Step 806: Concatenate the high-dimensional feature vectors before and after editing into a conditional vector, and input it into the DiT (Diffusion Transformer) based generative network module.

[0132] The generative network module maps a Gaussian distribution to a feature vector distribution, learns the "difference" between features before and after editing, and resolves this difference as fine-grained mesh modifications. For example, for the addition of a backpack, the conditional vector encodes information about structural changes in the back region.

[0133] Step 807: Generate the updated high-dimensional feature vector from the network output and input it into the decoder part of the VAE.

[0134] The decoder generates a high-resolution 3D symbolic distance field (SDF) and texture map field based on this vector, and recovers the updated texture mesh representation through spatial sampling. For example, the generated backpack mesh is smoothly integrated onto the original cartoon cat body mesh, while updating the texture map to match the new structure. This enables real-time updates, displaying the "kitten with a backpack" mesh model within seconds of user interaction.

[0135] Step 808: The grid generation results are synchronously displayed on the grid display area for users to view.

[0136] Step 809: Continuously monitor user operations.

[0137] If the user makes further edits (such as adding a hat), repeat steps 804 to 808 to achieve real-time bidirectional mapping. Finally, the edited 3D texture mesh file is output for the user to save or use.

[0138] In this embodiment, an application interface that links the 3D mesh display area and the voxel operation area in real time allows users to view the rendering effect of the mesh model in real time while manipulating the voxel model, thus improving the user experience. By manipulating the voxel model of the 3D mesh model, the high barrier to entry for direct manipulation of the 3D mesh model is reduced. Modifications to the 3D mesh model can be achieved by editing the voxel model without requiring strong professional knowledge, simplifying the complexity of 3D mesh model editing operations. Real-time bidirectional mapping between the 3D mesh model and the voxel model further enhances the user experience, making the linked display of the 3D mesh model and the voxel model more intuitive.

[0139] Based on the same technical concept, embodiments of this application provide a code change analysis device, such as... Figure 9 As shown, the device 900 includes: Secondly, embodiments of this application provide a three-dimensional mesh model generation device 900, comprising: Display module 901 is used to display a first three-dimensional mesh model in the mesh display area of ​​the application interface and to display a first voxel model in the voxel operation area of ​​the application interface; the first voxel model represents a plurality of voxel units corresponding to the first three-dimensional mesh model. The response module 902 is configured to generate and display a second voxel model in response to an editing operation on the first voxel model in the voxel operation area. The update module 903 is used to display a second three-dimensional mesh model associated with the second voxel model in the mesh display area; the second three-dimensional mesh model is obtained by updating the first three-dimensional mesh model based on the differences between the first voxel model and the second voxel model.

[0140] Optionally, the display module 901 is specifically used for: Based on the first three-dimensional mesh model, determine the initial voxel model; Obtain image features of the first 3D mesh model from different perspectives; The image features from different perspectives are mapped to the initial voxel model to obtain the first voxel model, which is then displayed in the voxel operation area of ​​the application interface.

[0141] Optionally, the display module 901 is specifically used for: The three-dimensional mesh model is then watertightened. The three-dimensional space containing the watertightened three-dimensional mesh model is divided into a uniform voxel space; The initial voxel model is constructed based on the location occupied by the watertightened 3D mesh model in the voxel space.

[0142] Optionally, the display module 901 is specifically used for: Multiple two-dimensional images of the first three-dimensional mesh model were acquired from different perspectives; Obtain the two-dimensional feature vector of the two-dimensional image from each viewpoint; The image features of the first three-dimensional mesh model under different viewpoints are determined based on the two-dimensional feature vectors under each viewpoint.

[0143] Optionally, the initial voxel model is composed of multiple voxel units; each voxel unit in the initial voxel model has three-dimensional coordinate information and occupancy status information. Optionally, the display module 901 is specifically used for: Determine the voxel unit corresponding to the image features under any viewpoint in the voxel model; For any voxel unit in the voxel model, multiple image features projected onto the voxel unit are processed to obtain a multidimensional feature representation of the voxel unit, thereby obtaining a first voxel model; any voxel unit in the first voxel model has three-dimensional coordinate information, occupancy status information, and multidimensional feature representation.

[0144] Optionally, an editing component is provided in the voxel operation area; Optionally, the response module 902 is specifically used for: The response to the editing operation on the first voxel model in the voxel operation area includes: The editing type is determined based on the user's actions on the editing component; Collect the voxel units corresponding to the user's editing operations on the first voxel model; Record the editing type, the voxel unit corresponding to the editing operation, and the position information of the voxel unit.

[0145] Optionally, the update module 903 is specifically used for: The first voxel model and the second voxel model are respectively input into the encoder to obtain the first high-dimensional voxel features and the second high-dimensional voxel features; The first high-dimensional voxel feature and the second high-dimensional voxel feature are input into the three-dimensional generation model to obtain the differential mesh feature; The difference grid features are input into the decoder to obtain the difference grid; The first three-dimensional mesh model is updated based on the difference mesh to obtain the second three-dimensional mesh model.

[0146] Optionally, the encoder and the decoder are jointly trained; wherein, during the training process, a first supervision signal and a second supervision signal are set, the first supervision signal being used to determine the accuracy of the geometry of the 3D mesh model, and the second supervision signal being used to determine the accuracy of the mesh texture of the 3D mesh model.

[0147] Optionally, the encoder includes alternately stacked sparse convolutional layers and attention layers; the sparse convolutional layers are used to extract the geometry of the voxel model, and the attention layers are used to determine the overall structure of the voxel model. The decoder is used to predict the vertex position offset of each voxel and the connection probability of the face, and to generate the difference mesh based on the vertex position offset of each voxel and the connection probability of the face.

[0148] Optionally, the update module 903 is specifically used for: If the editing operation is an add operation, then based on the position information of the voxel unit corresponding to the editing operation, the difference mesh is added to the first 3D mesh model to obtain the second 3D mesh model, or If the editing operation is a deletion operation, then based on the position information of the voxel unit corresponding to the editing operation, the difference mesh is deleted from the first 3D mesh model to obtain the second 3D mesh model. If the editing operation is a modification operation, then the difference mesh is mapped to the first three-dimensional mesh model according to the position information of the voxel unit corresponding to the editing operation to obtain the second three-dimensional mesh model.

[0149] Based on the same technical concept, embodiments of this application provide a computer device, which may be a terminal or a server, such as... Figure 10 As shown, it includes at least one processor 1001 and a memory 1002 connected to at least one processor. In this embodiment, the specific connection medium between the processor 1001 and the memory 1002 is not limited. Figure 10 Taking the connection between processor 1001 and memory 1002 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0150] In this embodiment of the application, the memory 1002 stores instructions that can be executed by at least one processor 1001. By executing the instructions stored in the memory 1002, at least one processor 1001 can perform the steps included in the above-described three-dimensional mesh model generation method.

[0151] The processor 1001 is the control center of the computer device, capable of connecting various parts of the computer device via various interfaces and lines. It executes instructions stored in the memory 1002 and retrieves data stored in the memory 1002. Optionally, the processor 1001 may include one or more processing units. The processor 1001 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the processor 1001. In some embodiments, the processor 1001 and the memory 1002 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0152] The processor 1001 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0153] Memory 1002, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 1002 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory 1002 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures that can be accessed by a computer, but is not limited thereto. In the embodiments of this application, memory 1002 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0154] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device. When the program is run on the computer device, it causes the computer device to perform the steps of the above-described three-dimensional mesh model generation method.

[0155] Based on the same inventive concept, this application provides a computer program product, characterized in that the computer program product includes a computer program stored on a computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by a computer device, the computer device performs the steps of the above-described three-dimensional mesh model generation method.

[0156] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0157] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0158] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0159] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0160] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for generating a three-dimensional mesh model, characterized in that, include: The first 3D mesh model is displayed in the mesh display area of ​​the application interface, and the first voxel model is also displayed in the voxel operation area of ​​the application interface. The first voxel model represents the corresponding multiple voxel units in the first three-dimensional mesh model; In response to an editing operation on the first voxel model in the voxel operation area, a second voxel model is generated and displayed; A second three-dimensional mesh model associated with the second voxel model is displayed in the mesh display area; the second three-dimensional mesh model is obtained by updating the first three-dimensional mesh model based on the differences between the first voxel model and the second voxel model.

2. The method as described in claim 1, characterized in that, Displaying the first voxel model in the voxel operation area of ​​the application interface includes: Based on the first three-dimensional mesh model, determine the initial voxel model; Obtain image features of the first 3D mesh model from different viewpoints; The image features from different perspectives are mapped to the initial voxel model to obtain the first voxel model, which is then displayed in the voxel operation area of ​​the application interface.

3. The method as described in claim 2, characterized in that, The step of determining the initial voxel model based on the first three-dimensional mesh model includes: The three-dimensional mesh model is then watertightened. The three-dimensional space containing the watertightened three-dimensional mesh model is divided into a uniform voxel space; The initial voxel model is constructed based on the location occupied by the watertightened 3D mesh model in the voxel space.

4. The method as described in claim 2, characterized in that, The step of obtaining image features of the first 3D mesh model from different viewpoints includes: Multiple two-dimensional images of the first three-dimensional mesh model were acquired from different perspectives; Obtain the two-dimensional feature vector of the two-dimensional image from each viewpoint; The image features of the first three-dimensional mesh model under different viewpoints are determined based on the two-dimensional feature vectors under each viewpoint.

5. The method as described in claim 2, characterized in that, The initial voxel model is composed of multiple voxel units; each voxel unit in the initial voxel model has three-dimensional coordinate information and occupancy status information. The step of mapping the image features from different viewpoints to the initial voxel model to obtain the first voxel model includes: Determine the voxel unit corresponding to the image features under any viewpoint in the voxel model; For any voxel unit in the voxel model, multiple image features projected onto the voxel unit are processed to obtain a multidimensional feature representation of the voxel unit, thereby obtaining a first voxel model; any voxel unit in the first voxel model has three-dimensional coordinate information, occupancy status information, and multidimensional feature representation.

6. The method as described in claim 1, characterized in that, An editing component is provided in the voxel operation area; the response to the editing operation on the first voxel model in the voxel operation area includes: The editing type is determined based on the user's actions on the editing component; Collect the voxel units corresponding to the user's editing operations on the first voxel model; Record the editing type, the voxel unit corresponding to the editing operation, and the position information of the voxel unit.

7. The method as described in any one of claims 1 to 6, characterized in that, The second 3D mesh model is obtained by updating the first 3D mesh model based on the differences between the first voxel model and the second voxel model, including: The first voxel model and the second voxel model are respectively input into the encoder to obtain the first high-dimensional voxel features and the second high-dimensional voxel features; The first high-dimensional voxel feature and the second high-dimensional voxel feature are input into the three-dimensional generation model to obtain the differential mesh feature; The difference grid features are input into the decoder to obtain the difference grid; The first three-dimensional mesh model is updated based on the difference mesh to obtain the second three-dimensional mesh model.

8. The method as described in claim 7, characterized in that, The encoder and the decoder are jointly trained; wherein, during the training process, a first supervision signal and a second supervision signal are set, the first supervision signal is used to determine the accuracy of the geometry of the 3D mesh model, and the second supervision signal is used to determine the accuracy of the mesh texture of the 3D mesh model.

9. The method as described in claim 7, characterized in that, The encoder includes alternating stacked sparse convolutional layers and attention layers; the sparse convolutional layers are used to extract the geometry of the voxel model, and the attention layers are used to determine the overall structure of the voxel model. The decoder is used to predict the vertex position offset of each voxel and the connection probability of the face, and to generate the difference mesh based on the vertex position offset of each voxel and the connection probability of the face.

10. The method as described in claim 7, characterized in that, The step of updating the first 3D mesh model based on the difference mesh to obtain the second 3D mesh model includes: If the editing operation is an add operation, then based on the position information of the voxel unit corresponding to the editing operation, the difference mesh is added to the first 3D mesh model to obtain the second 3D mesh model, or If the editing operation is a deletion operation, then based on the position information of the voxel unit corresponding to the editing operation, the difference mesh is deleted from the first 3D mesh model to obtain the second 3D mesh model. If the editing operation is a modification operation, then the difference mesh is mapped to the first three-dimensional mesh model according to the position information of the voxel unit corresponding to the editing operation to obtain the second three-dimensional mesh model.

Images