Reconstruction method and apparatus for three-dimensional entity model, device, medium and program product

Abstract

A reconstruction method and apparatus for a three-dimensional entity model, a device, a medium and a program product. The method comprises: obtaining three-dimensional space information corresponding to a three-dimensional entity model; on the basis of the three-dimensional space information, carrying out voxel division on the three-dimensional entity model, and determining voxels corresponding to the three-dimensional entity model and connected in a tree shape, wherein the voxels are distributed on the surface of the three-dimensional entity model, and used for representing the geometric shape of the surface of the three-dimensional entity model; on the basis of the voxels connected in the tree shape, constructing a three-dimensional mesh model corresponding to the three-dimensional entity model, wherein the three-dimensional mesh model is used for reconstructing a three-dimensional entity model after rendering.

Description

Three-dimensional solid model reconstruction method, device, equipment, medium and program product

[0001] This application claims priority to Chinese patent application No. 202311416241.X filed on October 30, 2023, entitled “Method, device, equipment, medium and program product for reconstructing three-dimensional solid models,” the entire contents of which are incorporated herein by reference. Technical Field

[0002] The present application relates to the field of computer vision, and in particular to a method, apparatus, device, medium and program product for reconstructing a three-dimensional solid model. Background Art

[0003] 3D model reconstruction technology can transform 3D physical models into 3D digital models, which can be easily stored, edited, analyzed, and transmitted. 3D model reconstruction technology is widely used in game rendering, augmented reality (AR), virtual reality (VR), and generative artificial intelligence (AIGC).

[0004] In related technologies, a deep learning surface modeling (Deep Marching Tetrahedra, DMTet) method is used to represent the surface of a three-dimensional solid model with a deformable tetrahedral mesh (Mesh), and to convert the signed distance value into a three-dimensional mesh model representation corresponding to the three-dimensional solid model.

[0005] However, related technologies generate sharp and narrow tetrahedral meshes when performing multiple subdivisions of the tetrahedral mesh near the surface of the three-dimensional solid model, which makes the three-dimensional mesh model inaccurate and causes the three-dimensional solid model to be unable to be accurately reconstructed.

[0006] Summary of the Invention

[0007] This application provides a method, apparatus, device, medium, and program product for reconstructing a three-dimensional solid model. The technical solution is as follows:

[0008] In one aspect, the present application provides a method for reconstructing a three-dimensional solid model, the method being executed by a computer device, the method comprising:

[0009] Obtaining three-dimensional space information corresponding to the three-dimensional solid model;

[0010] Based on the three-dimensional spatial information, the three-dimensional solid model is subjected to voxel division to determine voxels corresponding to the three-dimensional solid model that are connected in a tree-like manner; the voxels are distributed on a surface of the three-dimensional solid model to represent a geometric shape of the surface of the three-dimensional solid model;

[0011] A three-dimensional mesh model corresponding to the three-dimensional solid model is constructed based on the voxels connected in a tree shape, and the three-dimensional mesh model is used to reconstruct the three-dimensional solid model after rendering.

[0012] On the other hand, the present application provides a device for reconstructing a three-dimensional solid model, the device comprising:

[0013] An acquisition module is used to obtain three-dimensional space information corresponding to the three-dimensional entity model;

[0014] a partitioning module, configured to perform voxel partitioning on the three-dimensional solid model based on the three-dimensional spatial information, and determine voxels corresponding to the three-dimensional solid model that are connected in a tree-like manner; wherein the voxels are distributed on a surface of the three-dimensional solid model and are used to represent a geometric shape of the surface of the three-dimensional solid model;

[0015] A construction module is used to construct a three-dimensional mesh model corresponding to the three-dimensional solid model based on the voxels connected in a tree shape, and the three-dimensional mesh model is used to reconstruct the three-dimensional solid model after rendering.

[0016] On the other hand, the present application provides a computer device, comprising: a processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the three-dimensional solid model reconstruction method as described above.

[0017] On the other hand, the present application provides a computer-readable storage medium storing a computer program, which is loaded and executed by a processor to implement the three-dimensional solid model reconstruction method as described above.

[0018] On the other hand, the present application provides a computer program product, which includes computer instructions, which are stored in a computer-readable storage medium. A processor obtains the computer instructions from the computer-readable storage medium, so that the processor loads and executes them to implement the three-dimensional solid model reconstruction method as described above.

[0019] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0020] The present application provides a method for reconstructing a three-dimensional solid model, the method comprising: obtaining three-dimensional spatial information corresponding to the three-dimensional solid model by a computer device; performing voxel division on the three-dimensional solid model based on the three-dimensional spatial information to determine the voxels corresponding to the three-dimensional solid model that are connected in a tree-like manner; distributing the voxels on the surface of the three-dimensional solid model to characterize the surface geometry of the three-dimensional solid model; and constructing a three-dimensional mesh model corresponding to the three-dimensional solid model based on the voxels connected in the tree-like manner, wherein the three-dimensional mesh model is used to reconstruct the three-dimensional solid model after rendering. Accordingly, by performing voxel division on the three-dimensional solid model, compared to the deformable tetrahedral mesh used in related art, sharp and narrow tetrahedral meshes are not generated during fine division, making the surface geometry of the three-dimensional mesh model smoother and avoiding erroneous protrusions. That is, the surface geometry of the three-dimensional solid model can be accurately and meticulously characterized by each voxel, thereby accurately reconstructing the three-dimensional solid model based on the three-dimensional mesh model. The method can be widely used in fields such as gaming, rendering, AR / VR, 3D reconstruction, 3D-AIGC, 3D point cloud completion, and new view generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] FIG1 shows a block diagram of a computer system provided by an exemplary embodiment;

[0022] FIG2 is a schematic diagram showing a method for reconstructing a three-dimensional model provided by an exemplary embodiment;

[0023] FIG3 shows a flow chart of a method for reconstructing a three-dimensional model provided by an exemplary embodiment;

[0024] FIG4 shows a flow chart of a method for reconstructing a three-dimensional model provided by an exemplary embodiment;

[0025] FIG5 shows a flow chart of a method for reconstructing a three-dimensional model provided by an exemplary embodiment;

[0026] FIG6 shows a flow chart of a method for reconstructing a three-dimensional model provided by an exemplary embodiment;

[0027] FIG7 shows a flow chart of a method for reconstructing a three-dimensional model provided by an exemplary embodiment;

[0028] FIG8 shows an overall framework diagram of a method for reconstructing a three-dimensional model provided by an exemplary embodiment;

[0029] FIG9 is a schematic diagram showing voxel distribution of a three-dimensional solid model provided by an exemplary embodiment;

[0030] FIG10 shows a schematic diagram of octree-based position coding provided by an exemplary embodiment;

[0031] FIG11 is a schematic diagram showing a method for reconstructing a three-dimensional solid model provided by an exemplary embodiment;

[0032] FIG12 is a schematic diagram showing a method for reconstructing a three-dimensional solid model provided by an exemplary embodiment;

[0033] FIG13 shows a block diagram of a device for reconstructing a three-dimensional solid model provided by an exemplary embodiment;

[0034] FIG14 shows a structural block diagram of a computer device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0035] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0036] It should be understood that although the terms first, second, etc. may be used in this application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, a first parameter may also be referred to as a second parameter, and similarly, a second parameter may also be referred to as a first parameter. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0037] It should be noted that before collecting relevant user data (such as data of a three-dimensional solid model related to the user) and during the process of collecting relevant user data, this application can display a prompt interface, pop-up window or output a voice prompt message. The prompt interface, pop-up window or voice prompt message is used to remind the user that its relevant data is currently being collected, so that this application only starts to execute the relevant steps of obtaining user-related data after obtaining the user's confirmation operation on the prompt interface or pop-up window. Otherwise (that is, when the user's confirmation operation on the prompt interface or pop-up window is not obtained), the relevant steps of obtaining user-related data are terminated, that is, the user's relevant data is not obtained. In other words, all user data collected by this application are collected with the user's consent and authorization, and the collection, use and processing of relevant user data need to comply with the relevant laws, regulations and standards of relevant countries and regions.

[0038] First, a brief introduction to the terms involved in the embodiments of this application is given:

[0039] Octree: A tree-like data structure used to describe three-dimensional space. An octree recursively divides space into eight equal cubic subregions until a termination condition is reached. Each octree node represents a cubic volume element. Each node has eight child nodes, and the sum of the volume elements represented by the eight child nodes equals the volume of the parent node. Octrees enable efficient querying, insertion, and deletion of objects in space.

[0040] Signed Distance Field (SDF): A data structure used to represent and manipulate geometric shapes. An SDF is a scalar field that maps each point in space to a real value representing the signed distance from that point to the surface of the geometric shape. This real value is also called a signed distance value, and a signed distance value is a one-dimensional floating-point number. The SDF value for points on the surface of a shape is 0, the SDF value for points inside the shape is negative, and the SDF value for points outside the shape is positive.

[0041] Voxel: A combination of the words volume and pixel. A voxel can be thought of as a pixel in three-dimensional space, the smallest unit of 3D spatial segmentation. Voxels represent spatial units, have a specific size and position, and can be used to store specific properties. Voxels are widely used in computer vision fields such as 3D imaging, scientific data, and medical imaging.

[0042] Marching Cubes (MC) algorithm: A computer graphics algorithm used to generate three-dimensional model surfaces. It is primarily used to extract isosurfaces from three-dimensional scalar fields and can generate relatively accurate and smooth surfaces.

[0043] FIG1 shows a block diagram of a computer system 100 according to an exemplary embodiment of the present invention. The computer system 100 can be implemented as a system architecture for a method for reconstructing a three-dimensional solid model. The computer system 100 includes a terminal 120 and a server 140 .

[0044] The terminal 120 can be an electronic device such as a mobile phone, a tablet computer, a vehicle-mounted terminal (vehicle computer), a wearable device, a PC (Personal Computer), an unmanned reservation terminal, etc. A client that runs a target application can be installed in the terminal 120. The target application can be an application for three-dimensional data processing, display, reconstruction, and rendering of three-dimensional mesh models of three-dimensional solid models, or it can be other applications that provide three-dimensional data processing functions, display functions, reconstruction functions, and rendering functions of three-dimensional mesh models of three-dimensional solid models. This application does not limit this. In addition, this application does not limit the form of the target application, including but not limited to App (Application, application), mini-programs, etc. installed in the terminal 120, and can also be in the form of a web page.

[0045] Server 140 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud computing services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Server 140 can be the backend server of the target application described above, used to provide backend services to the client of the target application.

[0046] Cloud technology refers to a hosting technology that unifies hardware, software, and network resources within a wide area network (WAN) or local area network (LAN) to enable data computing, storage, processing, and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, and application technology, all based on the cloud computing business model. It can form a resource pool that can be used on demand with flexibility and convenience. Cloud computing technology will become a crucial support. Backend services for technical network systems, such as video websites, image websites, and more portals, require extensive computing and storage resources. With the rapid development and application of the internet industry, every item will likely have its own unique identification mark and will need to be transmitted to backend systems for logical processing. Data of varying levels will be processed separately, and data from all industries will require a strong system backend, which can only be achieved through cloud computing.

[0047] In some embodiments, the server 140 can also be implemented as a node in a blockchain system. Blockchain is a novel application model for computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Blockchain is essentially a decentralized database, a series of data blocks generated using cryptographic methods. Each data block contains information about a batch of network transactions, which is used to verify the validity of the information (for anti-counterfeiting) and generate the next block. Blockchain can include a blockchain underlying platform, a platform product service layer, and an application service layer.

[0048] The terminal 120 and the server 140 may communicate with each other via a network, such as a wired or wireless network.

[0049] In the method for reconstructing a three-dimensional solid model provided in the embodiments of the present application, each step may be performed by a computer device, which refers to an electronic device capable of computing, processing, and storing data. Taking the implementation environment of the solution shown in FIG1 as an example, the method for reconstructing a three-dimensional solid model may be performed by terminal 120 (for example, a client of a target application installed and running on terminal 120 may perform the method for reconstructing a three-dimensional solid model), by server 140, or by the interaction and cooperation of terminal 120 and server 140, which is not limited in this application.

[0050] Those skilled in the art will appreciate that the number of the terminals 120 may be greater or less. For example, there may be only one terminal 120, or there may be dozens, hundreds, or more of the terminals 120. The present embodiment does not limit the number or device type of the terminals 120.

[0051] Related technologies use a deep learning surface modeling method called Deep Marching Tetrahedra (DMTet). This method represents the surface of a 3D solid model as a deformable tetrahedral mesh and converts the signed distance values ​​into a 3D mesh representation of the 3D solid model. DMTet can then use the signed distance values ​​to perform subdivisions near the surface of the 3D solid model.

[0052] However, when the tetrahedral mesh is subdivided multiple times near the surface of a three-dimensional solid model, sharp and narrow tetrahedral meshes will be generated, and it is impossible to merge areas far away from the surface. Specifically, when the tetrahedral mesh is subdivided many times, a sharp and narrow tetrahedral mesh will be generated, which is reflected in the two-dimensional image as narrow and long triangular facets. Such triangular facets will cause the shape of the three-dimensional mesh model to be uneven and produce erroneous protrusions. Based on the uneven structure of the tetrahedral mesh, if it is directly merged, "inappropriate" ("No Proper") geometry will appear, that is, the vertices P of some triangular facets will appear on the boundaries of other triangular facets. When these vertices P are displaced, the related triangular facets will be staggered. Since it is not possible to merge and subdivide multiple times, the erroneous subdivision caused by inaccurate signed distance values ​​at low resolutions cannot be corrected, causing the three-dimensional mesh to be incorrectly aggregated to areas far away from the surface of the three-dimensional solid model.

[0053] In related technologies, an implicit neural network three-dimensional expression method based on signed distance values ​​(NeuS method) can also be used. This method combines implicit signed distance value representation with an unbiased volume rendering function, and redefines the opacity value so that the weight is maximized exactly on the zero level set surface of the signed distance value, thereby reconstructing an accurate and smooth surface through multi-view images.

[0054] However, this method is essentially based on volume rendering, and it represents the signed distance field in space using a large multilayer perceptron (MLP) model, which can express the mapping from spatial points to signed distance values. Both volume rendering and large MLPs slow training and inference speeds, and reconstructing a single 3D solid model can take hours, making 3D solid model reconstruction time-consuming and inefficient.

[0055] This embodiment provides a method for reconstructing a three-dimensional solid model. This method can subdivide and merge voxels of a three-dimensional solid model based on an octree, eliminating "No Proper" geometry and ensuring that the voxels remain regular cubes after multiple subdivisions. This ensures that the voxels are concentrated near the surface of the three-dimensional solid model, enabling detailed representation of the surface geometry of the three-dimensional solid model using a small number of voxels. Furthermore, the octree facilitates the maintenance of the tree structure and voxel parent-child inheritance relationships, thereby improving the accuracy of the extracted three-dimensional mesh model and facilitating the reconstruction of the three-dimensional solid model.

[0056] Fig. 2 is a schematic diagram of a method for reconstructing a three-dimensional solid model provided by an exemplary embodiment of the present application, wherein the method is executed by a computer device, and the computer device is a server 140 as an example for explanation.

[0057] Specifically, as shown in (1) of FIG2 , a series of images are captured around the three-dimensional solid model 141 by the camera 142. As shown in (2) of FIG2 , the server 140 obtains the series of images captured around the three-dimensional solid model 141 and the shooting posture of the camera 142 corresponding to the images, wherein the images carry at least one three-dimensional spatial information of depth value and point cloud data, so that the server 140 can obtain the three-dimensional spatial information of the three-dimensional solid model 141. Based on the three-dimensional spatial information, the server 140 performs voxel division on the three-dimensional solid model 141 and determines the voxels 143 corresponding to the three-dimensional solid model 141 that are connected in a tree-like manner; as shown in (3) of FIG2 , the voxels 143 are distributed on the surface of the three-dimensional solid model 141 and are used to represent the geometric shape of the surface of the three-dimensional solid model 141. At a position close to the surface of the three-dimensional solid model 141, the voxels 143 are more concentrated, smaller in volume, and finer in granularity. At a position far from the surface of the three-dimensional solid model 141, the voxels 143 are more dispersed, larger in volume, and coarser in granularity.

[0058] The server 140 constructs a three-dimensional mesh model corresponding to the three-dimensional solid model 141 based on the voxels connected in a tree-like manner. Optionally, the server 140 constructs a dual mesh corresponding to the three-dimensional solid model 141 based on the dual vertices corresponding to the voxels 143. The dual vertices are points that have duality with the voxels. The dual mesh is used to represent the surface geometry of the three-dimensional solid model 141. The server 140 transforms the dual mesh to obtain a three-dimensional mesh model corresponding to the three-dimensional solid model 141. The three-dimensional mesh model is used to reconstruct the three-dimensional solid model after rendering. As shown in (4) of Figure 2, the reconstructed three-dimensional solid model 144 is highly similar to the original three-dimensional solid model 141 and has a high degree of restoration.

[0059] In summary, the above scheme can extract a three-dimensional mesh model with high precision and restoration, so as to accurately reconstruct the three-dimensional solid model. The above scheme can be applied to the fields of game rendering, augmented reality, virtual reality (AR), virtual reality (VR), and generative artificial intelligence (AIGC). For example, the generated three-dimensional mesh model can be added to the game pipeline as a three-dimensional asset, and generate skinning, driving, animation and other effects. The three-dimensional mesh model can be rendered with a new arbitrary perspective to achieve immersive browsing and experience of AR / VR, thereby improving the user experience.

[0060] FIG3 shows a flowchart of a method for reconstructing a three-dimensional solid model provided by an exemplary embodiment of the present application. The method is described by applying the method to a computer device, which may be the terminal 120 and the server 140 shown in FIG1 . The method includes steps 220, 240, and 260:

[0061] Step 220: Acquire three-dimensional space information corresponding to the three-dimensional entity model.

[0062] The three-dimensional solid model refers to a three-dimensional model in a three-dimensional space that needs to be reconstructed.

[0063] Optionally, the 3D entity model may be a 3D entity in the real world or a fictitious 3D entity in the virtual world. For example, the 3D entity model may be at least one of a skull, a torso, a terrain, a building, a virtual skull of a virtual character, a virtual torso, a virtual terrain, and a virtual building.

[0064] The three-dimensional space information is information used to represent at least one of the geometric shape, size, volume, and color of a three-dimensional entity model in three-dimensional space.

[0065] Optionally, a camera is used to capture the 3D solid model in advance using a plurality of shooting postures to obtain a series of images corresponding to the 3D solid model. The images include 3D spatial information, such as at least one of depth values ​​and point cloud data. The computer device obtains the series of images and the shooting postures of the camera corresponding to the images to obtain the 3D spatial information corresponding to the 3D solid model.

[0066] Optionally, the shooting device includes various types of depth cameras, three-dimensional cameras, depth cameras, depth motion cameras, etc., and this embodiment does not limit the type of the shooting device. The multiple shooting postures include at least one of multiple shooting angles (e.g., at least one of a roll angle, a pitch angle, and a heading angle), a shooting position (e.g., at least one of a longitude, a latitude, and an altitude), and a shooting speed (e.g., at least one of a longitudinal speed, a lateral speed, and a vertical speed).

[0067] In step 240 , based on the three-dimensional spatial information, the three-dimensional solid model is divided into voxels to determine the tree-connected voxels corresponding to the three-dimensional solid model; each voxel is distributed on the surface of the three-dimensional solid model to represent the surface geometry of the three-dimensional solid model.

[0068] A voxel is a unit of space used to divide a 3D solid model. For example, a voxel is a regular cube; in different implementations, voxels can be implemented as other 3D shapes, such as spheres, hexagonal prisms, and cuboids. Optionally, voxels in 3D space can be spatially tessellated.

[0069] Optionally, the computer device divides the three-dimensional solid model into voxels using an octree method based on the three-dimensional spatial information, and determines the voxels corresponding to the three-dimensional solid model that are connected in a tree-like manner.

[0070] It should be noted that the volume and distribution of each voxel in the 3D solid model obtained by segmentation in this embodiment are non-uniform. Voxels are concentrated on the surface of the 3D solid model and are used to represent the surface geometry of the 3D solid model. Voxels closer to the surface of the 3D solid model have smaller volumes, more concentrated distribution, and finer granularity. Voxels further from the surface of the 3D solid model have larger volumes, less concentrated distribution, and coarser granularity.

[0071] In some embodiments, the positional relationship between each voxel and the surface of the three-dimensional solid model is determined by the signed distance value (SDF value) from the vertex of each voxel to the nearest surface of the three-dimensional solid model. The signed distance value of the vertex of each voxel can be encoded to form a signed distance field. In the process of voxel division of the three-dimensional solid model, continuous division and optimization can be performed based on the signed distance value. This process is briefly described as follows: the voxels located near the surface of the three-dimensional solid model are finely divided multiple times, and the voxels far from the surface of the three-dimensional solid model are merged. This merging can correct the erroneous subdivision at low resolution and reduce the overhead of uninteresting areas. The octree is further divided according to the divided octree. After several divisions, an octree composed of voxels with different volumes, uneven distribution, and concentrated on the surface of the three-dimensional solid model can be constructed. The geometric shape of the surface of the three-dimensional solid model can be carefully characterized by a small number of voxels.

[0072] Step 260 : constructing a three-dimensional mesh model corresponding to the three-dimensional solid model based on the voxels connected in a tree-like manner. The three-dimensional mesh model is used to reconstruct the three-dimensional solid model after rendering.

[0073] A 3D mesh model is a data structure used to represent a 3D solid model. It consists of a set of points, lines, and surfaces and is widely used in computer graphics.

[0074] Exemplarily, the computer device constructs a three-dimensional mesh model corresponding to the three-dimensional solid model based on the voxels connected in a tree-like manner. Exemplarily, the voxels connected in a tree-like manner are used to indicate the existence of voxels with at least two volumes, and the voxels with a larger volume can be split into multiple voxels with smaller volumes. Among the voxels connected in a tree-like manner, the three-dimensional shape composed of multiple voxels of smaller volume is the same as the three-dimensional shape of a voxel of larger volume. Taking the octree as an example, the volume elements represented by the 8 child nodes added together are equal to the volume of the parent node. It can be understood that in different implementations, the parent node can be split into more or fewer child nodes.

[0075] In summary, the method for reconstructing a three-dimensional solid model provided by an embodiment of the present application includes: a computer device obtains three-dimensional spatial information corresponding to the three-dimensional solid model; based on the three-dimensional spatial information, the three-dimensional solid model is voxel-divided to determine the voxels corresponding to the three-dimensional solid model that are connected in a tree-like manner; each voxel is distributed on the surface of the three-dimensional solid model to characterize the geometric shape of the surface of the three-dimensional solid model; based on each voxel connected in a tree-like manner, a three-dimensional mesh model corresponding to the three-dimensional solid model is constructed, and the three-dimensional mesh model is used to reconstruct the three-dimensional solid model after rendering. Accordingly, by voxel-dividing the three-dimensional solid model, the voxels connected in a tree-like manner in the three-dimensional mesh model can ensure that the voxels always maintain a regular three-dimensional shape. For example, when the voxels are cubes, the voxels in the three-dimensional mesh model always maintain a regular cube; compared with the deformable tetrahedral mesh used in the related art, no sharp and narrow tetrahedral mesh will appear during fine division, so that the geometric shape of the surface of the three-dimensional mesh model can be smoother and avoid the generation of erroneous protrusions. The geometric shape of the surface of the three-dimensional solid model can be accurately and meticulously characterized by each voxel. This method can also be widely used in games, rendering, AR / VR, 3D reconstruction, 3D-AIGC, 3D point cloud completion, new view generation and other fields.

[0076] The following embodiments provide a detailed description of the steps involved in the method for reconstructing a three-dimensional solid model.

[0077] Adaptive voxel partitioning

[0078] FIG4 shows a flow chart of a method for reconstructing a three-dimensional solid model provided by an exemplary embodiment of the present application. The above step 240 may be optionally replaced by steps 320 and 340:

[0079] In step 320 , the three-dimensional solid model is initialized based on the three-dimensional spatial information to obtain initialization voxels corresponding to the three-dimensional solid model that are connected in a tree-like manner.

[0080] Initialization voxels refer to voxels obtained by initializing a three-dimensional solid model.

[0081] For example, based on the three-dimensional spatial information, the three-dimensional solid model is initially divided into a low-resolution uniform octree, obtaining the initialization voxels corresponding to the three-dimensional solid model that are connected in a tree-like manner. The low resolution can be 16 bits or 32 bits. During the first initialization, the resolution of the octree is low and the initialization voxels are uniform, that is, the volume of each initialization voxel is the same and evenly distributed. As the subsequent iterative division continues, the resolution of the octree becomes higher and higher.

[0082] Step 340 : performing iterative adaptive partitioning on each initialized voxel based on the vertex attributes of each initialized voxel, and determining each voxel connected in a tree-like manner corresponding to the three-dimensional solid model.

[0083] Vertex attributes refer to the attributes of the vertices of each voxel.

[0084] In some embodiments, the vertex attributes include a signed distance value (SDF value) of the vertex, where the signed distance value of the i-th vertex is represented as si. Optionally, the vertex attributes also include one of the vertex's position coordinates vi and position code fi. The position coordinates vi are coordinates in the three-dimensional coordinate system corresponding to the three-dimensional solid model. The signed distance value si and the position code fi are optimizable parameters.

[0085] In this embodiment, by initializing the 3D solid model, the tree-connected initialization voxels corresponding to the 3D solid model can be obtained. This initialization can then be used to perform multiple iterative partitioning operations to determine the tree-connected voxels corresponding to the 3D solid model. Furthermore, this partitioning method can be applied as a plug-in to various optimization methods, such as implicit reconstruction based on 3D point clouds and SDF ground truth supervision, and multi-view reconstruction methods based on 2D image supervision and differentiable rendering.

[0086] Iterative processing

[0087] FIG5 shows a flow chart of a method for reconstructing a three-dimensional solid model provided by an exemplary embodiment of the present application. In some embodiments, the vertex attributes include signed distance values ​​of the vertices. The above step 340 is specifically implemented as steps 342, 344, 346, and 348:

[0088] Step 342: Determine each initialization voxel used in this iteration.

[0089] Step 344 : Count the signed distance values ​​of the vertices of each initialized voxel.

[0090] Specifically, since the present embodiment adopts an octree, each initialized voxel has 8 vertices, and statistics are calculated on the signed distance values ​​of the 8 vertices of each initialized voxel.

[0091] In step 346 , based on the signed distance values ​​of the vertices of each initialization voxel, the subdividable voxels and the mergeable voxels in each initialization voxel are determined.

[0092] Mergeable voxels are voxels that can be merged and determined in this iteration. Subdividable voxels are voxels that can be subdivided and determined in this iteration. Mergeable voxels are determined based on a preset merging threshold, and subdividable voxels are determined based on a preset subdivision threshold.

[0093] In step 348, the mergeable voxels are merged, and the subdividable voxels obtained in this round of iteration are used as initialization voxels for the next round of iteration. Each initialization voxel is iteratively adaptively divided until the iteration is terminated, thereby obtaining the tree-connected voxels corresponding to the three-dimensional solid model.

[0094] Specifically, in subsequent iterations, the subdividable voxels need to be further divided, while the mergeable voxels do not need to be further divided. The subdividable voxels obtained in this iteration are used as the initialization voxels for the next iteration. The iteration is terminated when the termination condition is met, and the tree-like connected voxels corresponding to the 3D solid model are obtained.

[0095] In some embodiments, the termination condition for the iteration includes the update amount of each tree-connected voxel being less than an update amount threshold, or the update error or pixel error corresponding to each tree-connected voxel being less than an error threshold compared to the previous iteration. Exemplarily, by iteratively determining subdividable voxels and mergeable voxels, the three-dimensional shape of the voxel-based 3D mesh model is made closer to that of the 3D solid model, thereby reducing the difference between the 3D mesh model and the 3D solid model.

[0096] In some embodiments, the computer device determines the signed distance value of the vertex through a multilayer perceptron (MLP) model. Specifically, the server also determines the position coordinates vi of the vertex of each initialized voxel; the position coordinates are input into the multilayer perceptron model to obtain the signed distance value of the vertex of each initialized voxel. Among them, with the continuous iterative division of voxels, the MLP model is also continuously optimized. Since the subsequent embodiments continue to determine the signed distance value for a small number of subdividable voxels, rather than querying the signed distance value corresponding to each voxel in a dense three-dimensional space, and the total parameters of the octree + MLP model in this embodiment are less than the parameters of the large MLP of the NeuS algorithm in the related art, the MLP model of this embodiment is smaller in scale, faster in training speed, and shorter in training time.

[0097] Next, the subdividable voxels and the mergeable voxels in step 346 are further described.

[0098] In some embodiments, the subdividable voxels and the mergeable voxels are determined based on a parameter called the signed distance value of the vertex of the initialized voxel. The sign of the signed distance value is used to characterize the positional relationship between the vertex and the surface of the three-dimensional solid model, wherein a positive sign indicates that the vertex is located inside the shape of the three-dimensional solid model, and a negative sign indicates that the vertex is located outside the shape of the three-dimensional solid model. The absolute value of the signed distance value is used to characterize the distance between the vertex and the surface of the three-dimensional solid model. When the distance is 0, it indicates that the vertex is a point on the surface of the three-dimensional solid model. Based on this, the above step 346 can be specifically implemented as step 346A, step 346B, step 346C, and step 346D:

[0099] Step 346A: Based on the signed distance values ​​of the vertices of the initialization voxels, determine the minimum absolute value among the absolute values ​​corresponding to the signed distance values ​​of the vertices of the initialization voxels.

[0100] Optionally, based on the signed distance values ​​of the vertices of the initialization voxels, the absolute values ​​corresponding to the signed distance values ​​of the vertices of the initialization voxels are determined, and the minimum absolute value is determined from the absolute values.

[0101] For example, if the absolute value corresponding to the signed distance value of the vertex is the minimum absolute value, it means that the vertex is the vertex closest to the surface of the three-dimensional solid model. It is understandable that the vertex may be located outside or inside the shape of the surface of the three-dimensional solid model.

[0102] Step 346B: if the minimum absolute value is less than the subdivision threshold and the signed distance values ​​of at least two vertices in the initialization voxel where the vertex corresponding to the minimum absolute value is located have opposite signs, the initialization voxel is determined as a subdividable voxel.

[0103] The subdivision threshold is a preset threshold used to indicate that a voxel can be used as a subdividable voxel.

[0104] In one example, the subdivision threshold may be determined based on at least one of the resolution of the octree and the scale of the 3D solid model. Different subdivision thresholds may be set for different 3D solid models. Optionally, the subdivision threshold is denoted as Tsub.

[0105] For example, when the minimum absolute value is less than the subdivision threshold, it indicates that the vertex corresponding to the minimum absolute value is relatively close to the surface of the 3D solid model. Furthermore, when the signed distance values ​​of at least two vertices in the initialization voxel where the vertex corresponding to the minimum absolute value is located have opposite signs, it indicates that the initialization voxel passes through the surface of the 3D solid model, and a portion of the initialization voxel is located inside the 3D solid model, while another portion is located outside the 3D solid model; therefore, the initialization voxel is considered a subdividable voxel.

[0106] In other embodiments, the above-mentioned minimum absolute value is less than the subdivision threshold, and in the initialization voxel where the vertex corresponding to the minimum absolute value is located, there are at least two vertices whose signed distance values ​​have opposite signs. These two conditions can also satisfy at least one of them.

[0107] Specifically, if the minimum absolute value is less than the subdivision threshold, the initialization voxel where the vertex corresponding to the minimum absolute value is located is determined to be a subdividable voxel. And / or, if the signed distance values ​​of at least two vertices in the initialization voxel where the vertex corresponding to the minimum absolute value is located have opposite signs, the initialization voxel is determined to be a subdividable voxel.

[0108] In some embodiments, the signed distance values ​​of the vertices of a tessellable voxel satisfy the following formula:

[0109] Among them, sdf i Represents the signed distance value of the i-th vertex of the voxel; |sdf i | represents the absolute value of the signed distance value corresponding to the i-th vertex of the voxel; Represents the minimum absolute value among the absolute values ​​corresponding to the signed distance values ​​of the 8 vertices of the voxel; T sub Indicates the segmentation threshold; sign indicates the symbol; & indicates that it satisfies simultaneously; The signs of the signed distance values ​​indicating that a voxel exists between at least two vertices are opposite.

[0110] In step 346C, the other initialized voxels except the subdividable voxels are treated as non-subdividable voxels.

[0111] In step 346D, when the absolute value of the signed distance value of the vertex of the non-divisible voxel is greater than the merging threshold, the non-divisible voxel is determined as a merging voxel.

[0112] The merging threshold is a preset threshold used to indicate that a voxel can be used as a merging voxel.

[0113] In one example, similar to the subdivision threshold, the merge threshold can be determined based on at least one of the resolution of the octree and the scale of the 3D solid model. Different merge thresholds can be set for different 3D solid models. Optionally, the merge threshold is denoted as Tmerge.

[0114] Optionally, if the absolute value of the signed distance values ​​of the vertices of the unsubdividable voxel is greater than a merging threshold, the unsubdividable voxel is determined to be a merging voxel. The absolute value of the signed distance values ​​of the vertices of the unsubdividable voxel being greater than the merging threshold may mean that the absolute value of the signed distance values ​​of at least some of the vertices of the unsubdividable voxel is greater than the merging threshold, or may mean that the absolute value of the signed distance values ​​of all vertices (8 vertices) of the unsubdividable voxel is greater than the merging threshold.

[0115] In some embodiments, when the minimum absolute value of the signed distance values ​​of the vertices of the unsubdividable voxel is greater than a merging threshold, the unsubdividable voxel is determined to be a merging voxel.

[0116] In some embodiments, the signed distance values ​​of the vertices of the mergeable voxels satisfy the following formula:

[0117] Among them, sdf i Represents the signed distance value of the i-th vertex of the voxel; |sdf i | represents the absolute value of the signed distance value corresponding to the i-th vertex of the voxel; Represents the minimum absolute value among the absolute values ​​corresponding to the signed distance values ​​of the 8 vertices of the voxel; T merge Indicates the merge threshold.

[0118] Exemplarily, since the absolute value of the signed distance value of a vertex represents the distance between the vertex and the nearest surface, this embodiment can obtain a schematic diagram of the voxel distribution of the three-dimensional solid model as shown in Figure 9 based on the correct signed distance value. (1) in Figure 9 is a two-dimensional schematic diagram, in which the grid 30 represents the voxel and the contour 31 represents the surface of the three-dimensional solid model. (2) in Figure 9 is a three-dimensional schematic diagram. It can be seen from (1) and (2) in Figure 9 that the voxels iteratively divided in this embodiment are concentrated near the surface of the three-dimensional solid model and are uneven, so that the geometric shape of the surface of the three-dimensional solid model can be accurately expressed using fewer voxels and signed distance values.

[0119] The above embodiment provides a method for determining whether a voxel is mergeable or subdividable, facilitating the merging of mergeable voxels and further fine-dividing of subdividable voxels during the iterative partitioning process. This allows the computer device to focus its data processing on subdividable voxels, resulting in a concentrated distribution of each voxel on the surface of the three-dimensional mesh model. This facilitates the use of fewer voxels to represent the surface geometry of the three-dimensional solid model, thereby improving the accuracy and efficiency of determining the three-dimensional mesh model.

[0120] FIG6 shows a flow chart of a method for reconstructing a three-dimensional solid model provided by an exemplary embodiment of the present application. The above step 260 may be optionally implemented as steps 262 and 264:

[0121] Step 262 : constructing a dual mesh corresponding to the three-dimensional solid model based on the dual vertices corresponding to each voxel; the dual vertices are points that are dual to the voxels, and the dual mesh is used to represent the surface geometry of the three-dimensional solid model.

[0122] In mathematics, duality is the correspondence between seemingly different theories that lead to the same physical results. In this embodiment, dual vertices are points that have duality with voxels. It can also be understood that from "voxel" to "point", the corresponding dual vertex of the voxel can be obtained.

[0123] A dual grid is a grid determined based on dual vertices. Optionally, each voxel in this embodiment has a dual vertex. The method for determining dual vertices will be described separately below.

[0124] Exemplarily, the computer device constructs a dual mesh corresponding to the three-dimensional solid model based on the dual vertices corresponding to each voxel. The dual mesh is used to represent the geometric shape of the surface of the three-dimensional solid model.

[0125] It can be understood that logically, the grid structure of the dual grid must be a regular grid. For a dual vertex in the dual grid, there are always 8 adjacent dual vertices in three-dimensional space (4 adjacent dual vertices in two-dimensional space), among which the 8 adjacent dual vertices can include overlapping dual vertices.

[0126] Step 264 , transforming the dual mesh to obtain a three-dimensional mesh model corresponding to the three-dimensional solid model.

[0127] A 3D mesh model is a data structure used to represent a 3D solid model. It consists of a set of points, lines, and surfaces and is widely used in computer graphics.

[0128] Optionally, the computer device transforms the dual grid to obtain a three-dimensional grid model corresponding to the three-dimensional solid model, and the three-dimensional grid model is used to reconstruct the three-dimensional solid model after rendering.

[0129] Optionally, the dual grid is transformed by at least one of a marching cubes algorithm (MC algorithm) and a Lewiner marching cubes algorithm (MC33 algorithm), and the specific method can be selected according to actual technical needs.

[0130] In this embodiment, based on the dual vertices corresponding to each voxel, a dual mesh corresponding to the three-dimensional solid model is constructed, and the dual mesh is transformed to obtain a three-dimensional mesh model corresponding to the three-dimensional solid model. This process is differentiable, so that the extracted three-dimensional mesh model can be more accurate.

[0131] Build a dual grid

[0132] In some embodiments, after determining each voxel of the three-dimensional solid model, the computer device needs to extract the grid structure. Then, the computer device can generate a three-dimensional grid model corresponding to the three-dimensional solid model based on the grid structure. Then, the above step 262 can be optionally replaced by steps 420 and 440:

[0133] Step 420 : Determine the dual vertex corresponding to each voxel based on the position code of the vertex of each voxel.

[0134] This embodiment is based on the tree structure of the octree, where voxel vertices naturally have multiple layers of parent-child relationships. Vertex position encoding refers to the encoding of the voxel vertex, which can represent the vertex's position in the octree, the number of layers, and the multiple layers of parent-child relationships between other voxels in the tree structure.

[0135] A dual vertex is a point obtained by dualizing a voxel to a point. Optionally, each voxel corresponds to a dual vertex. Due to the duality between the dual vertex and the voxel, the two lead to the same physical result; thus, a voxel with a volume is represented by this dual vertex without a volume. The dual vertex of a voxel is a point that can represent the voxel.

[0136] The following embodiments illustrate two methods for determining dual vertices. In practical applications, one of them or a combination of the two can be selected for use.

[0137] ·Determination of dual vertex 1

[0138] In some embodiments, the above step 420 is specifically implemented as steps 421 and 422:

[0139] Step 421 : Determine the voxel center point of each voxel based on the position code of the vertex of each voxel.

[0140] In step 422 , the voxel center point of each voxel is determined as the dual vertex corresponding to each voxel.

[0141] Optionally, based on the position encoding of the vertices of each voxel, an internal vertex, i.e., the voxel center point of each voxel, is extracted from each voxel. The voxel center point of each voxel is determined as the dual vertex corresponding to each voxel. From the perspective of the dimensionality of the three-dimensional space, the center point of the voxel in the three-dimensional space is determined as the dual vertex that can be used to represent the voxel. The dual vertex represents the position of the voxel in the three-dimensional space.

[0142] · Method 2 for determining dual vertices

[0143] In some embodiments, step 420 is specifically implemented as steps 423 and 424:

[0144] Step 423 : Based on the position code of the vertex of each voxel, perform interpolation operation on the signed distance value of the vertex of each voxel to obtain the interpolation operation point of each voxel.

[0145] In step 424 , the interpolation operation point of each voxel is determined as the dual vertex corresponding to each voxel.

[0146] An interpolation point is a point predicted by an interpolation operation.

[0147] Optionally, the interpolation operation includes at least one of nearest neighbor interpolation, bilinear interpolation, and three-dimensional linear interpolation. Exemplarily, based on the position coding of the vertices of each voxel, the signed distance value of the vertex of each voxel is interpolated to obtain the interpolation operation point of each voxel, and the interpolation operation point of each voxel is determined as the dual vertex corresponding to each voxel. From the dimension of the three-dimensional solid model, the signed distance between the vertex of the voxel and the three-dimensional solid model can represent the positional relationship between the vertex and the three-dimensional solid model (such as whether the vertex is inside or outside the three-dimensional solid model, and the distance between the vertex and the three-dimensional solid model). The interpolation result of the signed distance between each vertex of the voxel and the three-dimensional solid model is determined as the dual vertex, which can characterize the positional relationship between the voxel and the three-dimensional solid model.

[0148] Step 440 , connect the dual vertex of each voxel with the dual vertex of at least one adjacent voxel to construct a dual mesh corresponding to the three-dimensional solid model; the adjacent voxel refers to another voxel that has a common vertex with the voxel.

[0149] When two voxels share a common vertex, they are considered neighbors. That is, a voxel's neighbors are those voxels that share a common vertex with it. In this embodiment, each voxel has at least one neighboring voxel. Because this embodiment uses an octree, a voxel can have a maximum of eight neighbors.

[0150] The dual grid is a grid structure formed by connecting dual vertices.

[0151] Optionally, the computer device connects the dual vertex of each voxel with the dual vertex of at least one adjacent voxel to construct a dual mesh corresponding to the three-dimensional solid model.

[0152] As an example, FIG11 shows a schematic diagram of a reconstruction method for a three-dimensional solid model provided by an exemplary embodiment of the present application. As shown in the two-dimensional schematic diagram of voxels in FIG11 (1), each square 32 represents a voxel. Due to the uneven subdivision granularity of the voxels in this embodiment, there may not always be 8 voxels adjacent to the vertex near the vertex on the octree, which corresponds to the fact that there may not always be 4 voxels adjacent to the vertex near the vertex in the two-dimensional schematic diagram. Therefore, this embodiment extracts internal vertices from each voxel in the octree. These internal vertices constitute the dual vertices 34 shown in FIG11 (2). Each voxel only needs one dual vertex 34. As shown in FIG11 (3), the dual vertex 34 is connected to the dual vertices 36, dual vertices 38, etc. of its adjacent voxels to form a dual grid as shown in FIG11 (3). The dual grid must be a regular grid. Logically, a dual vertex can always find 8 adjacent dual vertices, and can contain overlapping dual vertices.

[0153] The above embodiments provide multiple ways to determine dual vertices, which improves the flexibility of determining dual vertices. The above embodiments can also construct a dual mesh corresponding to the three-dimensional solid model, so that the subsequent process of extracting the three-dimensional mesh model is differentiable, thereby improving the accuracy of the three-dimensional mesh model.

[0154] Vertex position encoding

[0155] In some embodiments, FIG7 shows a flowchart of a method for reconstructing a three-dimensional solid model provided by an exemplary embodiment of the present application. Before step 420, it is necessary to determine the position code of the vertex of each voxel in order to subsequently determine the dual vertex. The method further includes steps 522, 524, 526, 528, 530, and 532:

[0156] Step 522 : Determine the parent voxel of each voxel based on the tree-like connection relationship between each voxel.

[0157] The tree-like connection relationship refers to the multi-layer parent-child relationship of each voxel in the tree structure of the octree.

[0158] The parent voxel is a voxel in the previous layer that has a parent-child relationship with a voxel in the current layer.

[0159] Optionally, the computer device determines the parent voxel of each voxel based on the tree-like connection relationship between each voxel.

[0160] FIG10 shows a schematic diagram of an octree-based position encoding provided by an exemplary embodiment of the present application. FIG10 is a two-dimensional schematic diagram of an octree, which includes three layers. In the two-dimensional schematic diagram, the "octree" is displayed as a "quadtree". The vertices of the voxels divided by the first layer are circles 24. The second layer division is continued for the lower right corner voxel divided by the first layer, and the vertices of the voxels divided by the second layer are triangles 28. The third layer division is continued for the upper right corner voxel divided by the second layer, and the vertices of the voxels divided by the third layer are squares 26. There is a parent-child relationship between the three layers of voxels. Taking the vertex X of a voxel in the second layer in the two-dimensional schematic diagram of FIG10 as an example, for this vertex X, all the voxels divided by the first layer are the parent voxels of this vertex X; all the voxels in the second layer corresponding to this vertex X can serve as the parent voxels corresponding to the vertices of the voxels divided by the next layer.

[0161] Step 524 : Determine the parent position code of the parent vertex of the parent voxel corresponding to the vertex of each voxel.

[0162] The parent position code refers to the position code corresponding to the parent vertex.

[0163] Exemplarily, the computer device determines the parent position code of the parent vertex of each voxel corresponding to the vertex of each voxel. A vertex of a voxel has a maximum of 8 parent vertices. The parent position code of the i-th parent vertex of a vertex is represented as

[0164] Step 526: Based on the weight of the father's position code, perform weighted summation on the father's position code to obtain the father's weighted code corresponding to the father's position code.

[0165] Optionally, the weight of the parent position encoding can be a value determined based on the signed distance value of the parent vertex, or a learnable value, or a custom value. The weight of the parent position encoding of the i-th parent vertex of a vertex is expressed as The weighted coding of the father is expressed as

[0166] Exemplarily, the computer device performs weighted summation on the father position code based on the weight of the father position code to obtain the father weighted code corresponding to the father position code.

[0167] Step 528 : Create a new code for the vertex of each voxel, and the new code is used to represent the vertex.

[0168] The newly created code is used to represent the code of the vertex that needs to be represented this time. It is represented as

[0169] Exemplarily, the computer device creates a new code for the vertex of each voxel, and the new code is used to represent the vertex.

[0170] Step 530 : Determine the placeholder code of the vertex of each voxel according to the shape of the parent weighted code and the shape of the newly created code. The placeholder code is used to maintain the shape of the vertex position code.

[0171] Optionally, the value of the placeholder code is 0. The placeholder code is placed at the end of the vertex position code and is used to maintain the shape of the vertex position code. For example, in this embodiment, the shape of the vertex position code of each layer is set to the product of the number of layers L and the layer code dimension M.

[0172] Exemplarily, the computer device determines the placeholder code of the vertex of each voxel according to the shape of the parent weighted code and the shape of the newly created code.

[0173] In step 532 , the parent weighted code, the newly created code, and the placeholder code corresponding to the vertex of each voxel are serially connected in an array to obtain the position code of the vertex of each voxel.

[0174] Optionally, the computer device uses an array concatenation function (Concatenate, Cat) to concatenate the parent weighted code, the new code and the placeholder code corresponding to the vertex of each voxel to obtain the position code of the vertex of each voxel. The position code corresponding to a vertex can be expressed as f l .

[0175] As an example, taking the two-dimensional schematic diagram of the three-layer octree shown in FIG10 as an example, for a vertex of a voxel in a certain layer, the vertex is weighted by superimposing the parent Your own new encoding The placeholder code obtains the position code of the vertex. For example, the parent vertex of the vertex corresponds to the parent weighted code It is expressed as follows:

[0176] in, The weighted sum of the position codes of the vertices from their parent voxels, the weight ω can be a signed distance value of the vertex, a learnable value, or a custom value. Taking the position code of the vertex X of the triangle in the second layer in Figure 10 as an example, its parent weighted code is the weighted sum of the parent vertices of the circle from the first layer, and a new code for the vertices of the triangle in the second layer is created. In order to keep the shape of each layer of code L (number of layers) * M (layer code dimension), the tail end is supplemented with placeholder codes (all 0). Accordingly, the position code of each vertex retains the multi-layer parent association relationship and constructs a smooth spatial feature. In general, the position code of the vertices of a certain layer of the octree is expressed as follows:

[0177] In this embodiment, the octree can conveniently provide multi-level position coding for the vertex of each voxel, which is conducive to constructing smooth spatial features and making the three-dimensional mesh model more accurate.

[0178] Extract 3D Mesh

[0179] In some embodiments, the above step 264 is specifically implemented as follows: the computer device uses the marching cube algorithm to map the isosurface of the dual grid based on the signed distance value of each grid vertex in the dual grid; based on the isosurface of the dual grid, a three-dimensional grid model corresponding to the three-dimensional solid model is generated.

[0180] Optionally, the marching cube algorithm in this embodiment is the Lewiner marching cube algorithm (MC33 algorithm). Specifically, the computer device uses the MC33 algorithm to map the signed distance values ​​of each mesh vertex in the dual mesh to obtain an isosurface of the dual mesh, based on the positive and negative signs of the signed distance values ​​of each mesh vertex in the dual mesh, wherein the signed distance value corresponding to the point on the isosurface is 0. Based on the isosurface of the dual mesh, a three-dimensional mesh model corresponding to the three-dimensional solid model can be generated. In some examples, the center points and / or interpolation points of the voxels in the three-dimensional mesh model can also be used as supplementary points to obtain a three-dimensional mesh model with higher density.

[0181] It should be noted that in a related art marching cubes (MC) algorithm, when extracting a three-dimensional mesh model, it is required to extract based on a regular grid, that is, each vertex has 8 voxels adjacent to the vertex. However, this embodiment processes the dual grid. As an example, FIG12 shows a schematic diagram of a three-dimensional solid model reconstruction method provided by an exemplary embodiment of the present application. Among them, (1-1) and (1-2) shown in (1) of FIG12 are two surface segmentation methods in the MC algorithm of the related art, and the black dots / white dots respectively represent the signed distance values ​​of the vertices as positive / negative. Due to the existence of these two surface segmentation methods, the extracted three-dimensional mesh model may have a geometric hole 40 shown in (2) of FIG12. Therefore, this embodiment provides a differentiable dual marching cube method, which uses the Lewiner marching cube algorithm (MC33 algorithm) to process the dual grid and extract the three-dimensional mesh model corresponding to the dual grid. Among them, the two-dimensional schematic diagram of the extracted three-dimensional mesh model is shown in (4) in Figure 11, and the three-dimensional schematic diagram of the three-dimensional mesh model is shown in (3) in Figure 2 and (3) in Figure 12. The three-dimensional mesh model does not have the geometric void 40 shown in (2) in Figure 12, ensuring the topological correctness and fluidity of the three-dimensional mesh model.

[0182] The three-dimensional mesh model extracted in this embodiment can effectively avoid the generation of geometric voids, thereby ensuring the topological correctness and fluidity of the three-dimensional mesh model.

[0183] The following describes the reconstruction method of the three-dimensional solid model provided by this embodiment in conjunction with the overall framework diagram. Figure 8 shows the overall framework diagram of the reconstruction method of the three-dimensional solid model provided by an exemplary embodiment of the present application. The overall framework diagram 10 can be briefly described as follows:

[0184] The 3D spatial information of the 3D solid model is initialized 11 and initially partitioned into a uniform octree with a low resolution (e.g., 16 / 32). The resulting uniform octree is shown in a two-dimensional diagram 12. The initial vertex position vi, vertex attributes si, and position code fi of each voxel in the octree are determined. The vertex attributes si and position code fi are both optimizable network parameters 13. Adaptive partitioning is performed based on the vertex attributes si 14. For each voxel, the SDF values ​​of its eight vertices are counted. If the minimum SDF absolute value is less than a subdivision threshold Tsub, and if at least two vertices have SDF values ​​of opposite signs, the voxel is identified as a subdividable voxel. Among the non-subdividable voxels, those whose eight vertices have SDF absolute values ​​greater than a merge threshold Tmerge are selected and identified as mergeable voxels. The octree is uniformly subdivided and merged based on the mergeable and subdividable voxels. The processed octree is shown in a two-dimensional diagram 15. After each adaptive partitioning, the vertex position vi, vertex attribute si, and position code fi of each vertex are reassigned based on the new octree to determine whether the current round of processing has converged 16. If not, a new round of optimization is performed. The optimization and partitioning cycle is repeated multiple times until the overall optimization converges.

[0185] After the optimization converges, a differentiable dual marching cubes method is used to extract a dual grid (Dual Grid) 17 based on each partitioned voxel. In related art, the marching cubes algorithm requires the input structure to be a regular grid, meaning that each vertex has eight adjacent cubes. However, due to the uneven granularity of the voxels in this embodiment, each vertex may not always have eight adjacent cubes. Therefore, this embodiment extracts internal vertices from each voxel. These internal vertices can be the voxel center or interpolated from the signed distance values ​​of the voxel's vertices. These internal vertices constitute the corresponding dual vertices (Dual Verts) of the voxel. The marching cubes algorithm places vertices on the edge of each cube, while the dual marching cubes method in this embodiment places vertices inside each voxel. Each voxel only requires one dual vertex. The dual vertex is connected to the dual vertices of its adjacent voxels to form the dual grid. The dual grid must be a regular grid. Logically, eight adjacent dual vertices can always be found for a dual vertex, which can include overlapping dual vertices. The dual grid is shown as a two-dimensional schematic diagram 18. Then, the MC33 algorithm is used to extract the three-dimensional grid model 19 corresponding to the dual grid. The three-dimensional grid model is shown as a two-dimensional schematic diagram 20 and a three-dimensional schematic diagram 21.

[0186] In summary, the method for reconstructing a three-dimensional solid model provided in this embodiment has at least the following beneficial effects:

[0187] 1. Based on a flexible way of dividing voxels, the voxels in the octree are concentrated near the surface of the 3D solid model, and the detailed shape of the 3D solid model is expressed with fewer voxels. Therefore, it is only necessary to infer the signed distance value near the surface rather than in the dense 3D space, which can accelerate data processing.

[0188] 2. Dividing voxels based on octree facilitates providing multi-level position encoding for voxel vertices, which is conducive to constructing smooth spatial features.

[0189] 3. Based on the voxels divided by the octree, dual vertices and dual meshes are extracted, and then a three-dimensional mesh model is extracted. This process is differentiable and can be widely used in model reconstruction based on differentiable rendering.

[0190] 4. The expression method of the three-dimensional mesh model provided in this embodiment can be widely used as a plug-in in games, rendering, AR / VR, 3D reconstruction, 3D-AIGC, 3D point cloud completion, new view generation and other fields.

[0191] FIG13 shows a block diagram of a 3D solid model reconstruction apparatus 800 provided by an exemplary embodiment of the present application. The 3D solid model reconstruction apparatus 800 includes:

[0192] An acquisition module 810 is configured to execute step 220 in the embodiment of FIG3 ;

[0193] A division module 820, configured to execute step 240 in the embodiment of FIG3;

[0194] The construction module 830 is used to execute step 260 in the embodiment of FIG. 3 .

[0195] In some embodiments, the partitioning module 820 is used to execute step 320 and step 340 in the embodiment of FIG. 4 .

[0196] In some embodiments, the vertex attribute comprises a signed distance value for the vertex.

[0197] In some embodiments, the partitioning module 820 is used to execute step 342, step 344, step 346, and step 348 in the embodiment of FIG. 4 .

[0198] In some embodiments, the sign of the signed distance value is used to represent the positional relationship between the vertex and the surface of the three-dimensional solid model, and the absolute value of the signed distance value is used to represent the distance between the vertex and the surface of the three-dimensional solid model.

[0199] In some embodiments, the partitioning module 820 is configured to: determine, based on the signed distance values ​​of the vertices of the initialization voxels, a minimum absolute value among the absolute values ​​corresponding to the signed distance values ​​of the vertices of the initialization voxels;

[0200] When the minimum absolute value is smaller than the subdivision threshold and, in the initialization voxel where the vertex corresponding to the minimum absolute value is located, there are at least two vertices whose signed distance values ​​have opposite signs, the initialization voxel is determined as the subdividable voxel;

[0201] Treating other initialized voxels except the subdividable voxels as non-subdividable voxels;

[0202] When the absolute value corresponding to the signed distance value of the vertex of the non-divisible voxel is greater than the merging threshold, the non-divisible voxel is determined as the mergable voxel.

[0203] In some embodiments, the apparatus further comprises a processing module; the processing module is configured to:

[0204] Determining the position coordinates of the vertices of each of the initialized voxels;

[0205] The position coordinates are input into a multi-layer perception model to obtain the signed distance values ​​of the vertices of the initialization voxels.

[0206] In some embodiments, the construction module 830 is used to execute step 262 and step 264 in the embodiment of FIG. 6 .

[0207] In some embodiments, the construction module 830 is used to execute step 420 and step 440 in the embodiment of FIG. 6 .

[0208] In some embodiments, the construction module 830 is used to:

[0209] Determining a voxel center point of each voxel based on the position code of the vertex of each voxel;

[0210] The voxel center point of each voxel is determined as the dual vertex corresponding to each voxel.

[0211] In some embodiments, the construction module 830 is configured to: perform an interpolation operation on the signed distance values ​​of the vertices of each voxel based on the position codes of the vertices of each voxel to obtain an interpolation operation point of each voxel;

[0212] The interpolation operation points of the respective voxels are determined as the dual vertices corresponding to the respective voxels.

[0213] In some embodiments, the apparatus further includes a processing module; the processing module is configured to execute steps 522 to 532 in the embodiment of FIG. 7 .

[0214] In some embodiments, the construction module 830 is configured to: use a marching cubes algorithm to map an isosurface of the dual mesh based on the signed distance values ​​of each mesh vertex in the dual mesh;

[0215] Based on the isosurface of the dual grid, a three-dimensional grid model corresponding to the three-dimensional solid model is generated.

[0216] It should be noted that the specific limitations of the embodiment of the apparatus 800 for reconstructing one or more three-dimensional solid models provided above can be found in the limitations of the three-dimensional solid model reconstruction method described above and will not be repeated here. Each module of the apparatus described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor of a computer device in the form of hardware, or may be stored in the form of software in a memory of the computer device so that the processor can call and execute the corresponding operations of each module.

[0217] An embodiment of the present application also provides a computer device, which includes: a processor and a memory, wherein a computer program is stored in the memory; the processor is used to execute the computer program in the memory to implement the three-dimensional solid model reconstruction method provided by the above-mentioned method embodiments.

[0218] 14 is a block diagram of a computer device 1000 according to an exemplary embodiment of the present application. Optionally, the computer device 1000 is a server 1000.

[0219] Typically, the server 1000 includes a processor 1001 and a memory 1002 .

[0220] The processor 1001 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 1001 may be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1001 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a central processing unit (CPU); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 1001 may be integrated with a graphics processing unit (GPU), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 1001 may also include an artificial intelligence (AI) processor, which is used to process computing operations related to machine learning.

[0221] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 1002 is used to store at least one instruction, which is executed by the processor 1001 to implement the three-dimensional solid model reconstruction method provided in the method embodiment of the present application.

[0222] In some embodiments, the server 1000 may further optionally include: an input interface 1003 and an output interface 1004. The processor 1001, the memory 1002, and the input interface 1003 and the output interface 1004 may be connected via a bus or a signal line. Each peripheral device may be connected to the input interface 1003 and the output interface 1004 via a bus, a signal line, or a circuit board. The input interface 1003 and the output interface 1004 may be used to connect at least one peripheral device related to input / output (I / O) to the processor 1001 and the memory 1002. In some embodiments, the processor 1001, the memory 1002, and the input interface 1003 and the output interface 1004 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 1001, the memory 1002, and the input interface 1003 and the output interface 1004 may be implemented on a separate chip or circuit board, which is not limited in the embodiments of the present application.

[0223] Those skilled in the art will appreciate that the structure shown in FIG. 14 does not limit the computer device 1000 , and may include more or fewer components than shown, or combine certain components, or adopt a different component arrangement.

[0224] In an exemplary embodiment, the present application provides a chip comprising a programmable logic circuit and / or program instructions, which, when run on a computer device, is used to implement the three-dimensional solid model reconstruction method provided in the above method embodiment.

[0225] The present application provides a computer-readable storage medium storing a computer program, which is loaded and executed by a processor to implement the three-dimensional solid model reconstruction method provided by the above method embodiment.

[0226] The present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the processor of the computer device to load and execute the computer instructions to implement the method for reconstructing a three-dimensional solid model provided in the above-mentioned method embodiment.

[0227] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0228] Those skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware, or may be accomplished by a program to instruct the relevant hardware, and the program may be stored in a computer-readable storage medium, which may be a read-only memory, a disk, or an optical disk, etc.

[0229] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0230] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for reconstructing a three-dimensional solid model, the method being executed by a computer device, the method comprising: Obtaining three-dimensional space information corresponding to the three-dimensional solid model; Based on the three-dimensional space information, the three-dimensional solid model is divided into voxels to determine the voxels connected in a tree shape corresponding to the three-dimensional solid model; The voxels are distributed on the surface of the three-dimensional solid model and are used to characterize the geometric shape of the surface of the three-dimensional solid model; Based on the voxels connected in a tree shape, a three-dimensional mesh model corresponding to the three-dimensional solid model is constructed, and the three-dimensional mesh model is used to reconstruct the three-dimensional solid model after rendering.

2. The method according to claim 1, wherein: The step of dividing the three-dimensional solid model into voxels based on the three-dimensional spatial information to determine the voxels corresponding to the three-dimensional solid model that are connected in a tree-like manner includes: Initializing the three-dimensional solid model based on the three-dimensional space information to obtain initialization voxels corresponding to the three-dimensional solid model that are connected in a tree shape; Based on the vertex attributes of the initialization voxels, the initialization voxels are iteratively and adaptively divided to determine the tree-connected voxels corresponding to the three-dimensional solid model.

3. The method according to claim 2, wherein: The vertex attributes include a signed distance value of the vertex; The iterative adaptive division of each of the initialization voxels based on the vertex attributes of each of the initialization voxels to determine each of the voxels connected in a tree-like manner corresponding to the three-dimensional solid model includes: Determine the initialization voxels used in this round of iteration; Counting the signed distance values ​​of the vertices of each of the initialized voxels; Determine the subdividable voxels and the mergeable voxels in each of the initialization voxels based on the signed distance values ​​of the vertices of each of the initialization voxels; The mergeable voxels are merged, and the subdividable voxels obtained in this round of iteration are used as initialization voxels for the next round of iteration. Each of the initialization voxels is iteratively adaptively divided until the iteration is terminated, so as to obtain the tree-connected voxels corresponding to the three-dimensional solid model.

4. The method according to claim 3, wherein: The sign of the signed distance value is used to represent the positional relationship between the vertex and the surface of the three-dimensional solid model, and the absolute value of the signed distance value is used to represent the distance between the vertex and the surface of the three-dimensional solid model; The determining, based on the signed distance values ​​of the vertices of the respective initialization voxels, of the subdividable voxels and the mergeable voxels in the respective initialization voxels comprises: Based on the signed distance values ​​of the vertices of the initialization voxels, determining the minimum absolute value among the absolute values ​​corresponding to the signed distance values ​​of the vertices of the initialization voxels; When the minimum absolute value is less than the subdivision threshold and in the initialization voxel where the vertex corresponding to the minimum absolute value is located, there are at least two vertices whose signed distance values ​​have different signs, the initialization voxel is determined as the subdividable voxel; Treating other initialized voxels except the subdividable voxels as non-subdividable voxels; When the absolute value corresponding to the signed distance value of the vertex of the non-divisible voxel is greater than the merging threshold, the non-divisible voxel is determined as the merging voxel.

5. The method according to claim 3, wherein: The method further comprises: Determining the position coordinates of the vertices of each of the initialized voxels; The position coordinates are input into a multi-layer perception model to obtain the signed distances of the vertices of each initialized voxel. value.

6. The method according to any one of claims 1 to 5, wherein: The constructing of a three-dimensional mesh model corresponding to the three-dimensional solid model based on the voxels connected in a tree shape includes: Based on the dual vertices corresponding to the voxels, constructing a dual mesh corresponding to the three-dimensional solid model; the dual vertices are points that have duality with the voxels, and the dual mesh is used to characterize the geometric shape of the surface of the three-dimensional solid model; The dual grid is transformed to obtain a three-dimensional grid model corresponding to the three-dimensional solid model.

7. The method according to claim 6, wherein: The step of constructing a dual mesh corresponding to the three-dimensional solid model based on the dual vertices corresponding to each voxel includes: Determine the dual vertex corresponding to each voxel based on the position code of the vertex of each voxel; The dual vertices of each voxel are connected to the dual vertices of at least one adjacent voxel to construct a dual mesh corresponding to the three-dimensional solid model; the adjacent voxels refer to other voxels that have common vertices with the voxel.

8. The method according to claim 7, wherein: The step of determining the dual vertices corresponding to each voxel based on the position coding of the vertices of each voxel comprises: Determining a voxel center point of each voxel based on the position code of the vertices of each voxel; The voxel center point of each voxel is determined as the dual vertex corresponding to each voxel.

9. The method according to claim 7, wherein: The step of determining the dual vertices corresponding to each voxel based on the position coding of the vertices of each voxel comprises: Based on the position codes of the vertices of each voxel, interpolation operations are performed on the signed distance values ​​of the vertices of each voxel to obtain interpolation operation points of each voxel; The interpolation operation points of the respective voxels are determined as the dual vertices corresponding to the respective voxels.

10. The method according to any one of claims 6 to 9, wherein: The method further comprises: Determine the parent voxel of each voxel based on the tree-like connection relationship between the voxels; Determine the father position code of the father vertex from the father voxel corresponding to the vertex of each voxel; Based on the weight of the father position code, weighted summing the father position code is performed to obtain the father weighted code corresponding to the father position code; Creating a new code for the vertex of each voxel, wherein the new code is used to represent the vertex; Determine, according to the shape of the father weighted code and the shape of the newly created code, the placeholder code of the vertex of each voxel, wherein the placeholder code is used to maintain the shape of the position code of the vertex; The parent weighted code, the newly created code and the placeholder code corresponding to the vertices of each voxel are serially connected in array to obtain the position code of the vertices of each voxel.

11. The method according to any one of claims 6 to 9, wherein: The step of transforming the dual grid to obtain a three-dimensional grid model corresponding to the three-dimensional solid model includes: Using a marching cube algorithm, based on the signed distance values ​​of each mesh vertex in the dual mesh, mapping obtains an isosurface of the dual mesh; Based on the isosurface of the dual mesh, a three-dimensional mesh model corresponding to the three-dimensional solid model is generated.

12. A device for reconstructing a three-dimensional solid model, the device comprising: An acquisition module, used to acquire three-dimensional space information corresponding to the three-dimensional entity model; A partitioning module is used to perform voxel partitioning on the three-dimensional solid model based on the three-dimensional space information to determine the The three-dimensional solid model corresponds to the individual voxels connected in a tree-like manner; The voxels are distributed on the surface of the three-dimensional solid model and are used to characterize the geometric shape of the surface of the three-dimensional solid model; A construction module is used to construct a three-dimensional mesh model corresponding to the three-dimensional solid model based on the voxels connected in a tree shape, and the three-dimensional mesh model is used to reconstruct the three-dimensional solid model after rendering.

13. A computer device, comprising: A processor and a memory, wherein the memory stores a computer program, and the computer program is loaded and executed by the processor to implement the three-dimensional solid model reconstruction method according to any one of claims 1 to 11.

14. A computer-readable storage medium storing a computer program, wherein the computer program is loaded and executed by a processor to implement the method for reconstructing a three-dimensional solid model as claimed in any one of claims 1 to 11.

15. A computer program product, comprising computer instructions, wherein the computer instructions are stored in a computer-readable storage medium, and a processor obtains the computer instructions from the computer-readable storage medium, so that the processor loads and executes the computer instructions to implement the method for reconstructing a three-dimensional solid model as described in any one of claims 1 to 11.

Images