Three-dimensional modeling method and device, electronic equipment and storage medium

Abstract

Embodiments of the invention provide a three-dimensional modeling method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining a voxel space of an object; the voxel space is used for representing a physical space where the object is located; dividing the voxel space into voxel units of a plurality of levels; wherein each level comprises a plurality of voxel units; each voxel unit comprises one or more voxels; each voxel unit of the first level is obtained by dividing the voxel space; the voxel units of the next level are located in the voxel units of the previous level, the voxel units of the next level are obtained by dividing specific voxel units in the voxel units of the previous level, and the specific voxel units comprise voxels located on the surface of the three-dimensional model of the object; the voxel is used for representing a space region in a physical space where the object is located; and establishing a three-dimensional model of the object based on the divided voxel units of the last level.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a three-dimensional modeling method, device, electronic device and storage medium. Background Art

[0002] 3D modeling technology has been widely used in various fields. For example, in the film and television industry, 3D modeling technology is used to generate virtual characters or objects to create better picture effects. In the game industry, 3D modeling technology is used to build game characters to bring players a better gaming experience. In the live broadcast industry, 3D modeling technology is used to generate virtual objects and realize live broadcast of virtual objects. However, the existing 3D modeling methods have the disadvantage of slow modeling speed when applied to the scene of real-time generation of 3D models. Summary of the invention

[0003] The embodiments of the present disclosure provide a 3D modeling method, device, electronic device and storage medium, which can be applied in a scenario of generating a 3D model in real time to improve the 3D modeling speed.

[0004] In a first aspect, an embodiment of the present disclosure provides a three-dimensional modeling method, comprising:

[0005] Acquire a voxel space of an object; the voxel space is used to represent a physical space where the object is located;

[0006] The voxel space is divided into a plurality of levels of voxel units; wherein each level includes a plurality of voxel units; each of the voxel units includes one or more voxels; each voxel unit of the first level is obtained by dividing the voxel space; the voxel units of the next level are located inside the voxel units of the previous level, and the voxel units of the next level are obtained by dividing specific voxel units in the voxel units of the previous level, and the specific voxel units include voxels located on the surface of the three-dimensional model of the object; the voxels are used to represent the spatial area in the physical space where the object is located;

[0007] A three-dimensional model of the object is established based on the voxel units at the last level of the division.

[0008] In a second aspect, an embodiment of the present disclosure provides a three-dimensional modeling device, including:

[0009] An acquisition unit, used for acquiring a voxel space of an object; the voxel space is used for representing a physical space where the object is located;

[0010] A division unit, used for dividing the voxel space into a plurality of levels of voxel units; wherein each level includes a plurality of voxel units; each of the voxel units includes one or more voxels; each voxel unit of the first level is obtained by dividing the voxel space; the voxel units of the next level are located inside the voxel units of the previous level, and the voxel units of the next level are obtained by dividing specific voxel units in the voxel units of the previous level, and the specific voxel units include voxels located on the surface of the three-dimensional model of the object; the voxels are used to represent the spatial area in the physical space where the object is located;

[0011] The modeling unit is used to establish a three-dimensional model of the object based on the voxel units at the last level of the division.

[0012] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a processor; and a memory configured to store computer-executable instructions, wherein the computer-executable instructions, when executed, enable the processor to implement the steps of the method described in the first aspect above.

[0013] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions implement the steps of the method described in the first aspect when executed by a processor.

[0014] In one or more embodiments of the present disclosure, a voxel space of an object is obtained, and the voxel space is used to represent the physical space where the object is located. The voxel space is divided into multiple levels of voxel units, wherein each level includes multiple voxel units; each voxel unit includes one or more voxels; each voxel unit of the first level is obtained by dividing the voxel space; the voxel units of the next level are located inside the voxel units of the previous level, and the voxel units of the next level are obtained by dividing specific voxel units in the voxel units of the previous level, and the specific voxel units include voxels located on the surface of the three-dimensional model of the object; voxels are used to represent the spatial area in the physical space where the object is located; based on the voxel units of the last level of division, a three-dimensional model of the object is established. It can be seen that through this embodiment, the voxel space can be divided into multiple levels of voxel units layer by layer, and the division is based on whether the voxel units of the previous level that have been divided include voxels located on the surface of the three-dimensional model of the object, and based on the voxel units of the last level of division, a three-dimensional model of the object is established. Therefore, when performing three-dimensional modeling, there is no need to pay attention to all the voxels in the voxel space, only the voxel units of the last level need to be paid attention to to achieve three-dimensional modeling, which reduces the amount of data used in the modeling process and improves the three-dimensional modeling speed in the scenario of real-time generation of three-dimensional models. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate one or more embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative labor.

[0016] Figure 1 A schematic diagram of a flow chart of a three-dimensional modeling method provided in one embodiment of the present disclosure;

[0017] Figure 2a A schematic diagram of the existing technology for dividing the voxel space;

[0018] Figure 2b A schematic diagram of dividing a voxel space provided by an embodiment of the present disclosure;

[0019] Figure 2c A schematic diagram of dividing a voxel space provided in accordance with another embodiment of the present disclosure;

[0020] Figure 3 A schematic diagram of the principle of determining whether a voxel unit at a previous level is a specific voxel unit provided by an embodiment of the present disclosure;

[0021] Figure 4 A schematic diagram of voxel space division and data storage provided by an embodiment of the present disclosure;

[0022] Figure 5 A schematic diagram of voxel expansion provided by an embodiment of the present disclosure;

[0023] Figure 6a A schematic diagram of the effect of three-dimensional reconstruction provided by an embodiment of the present disclosure;

[0024] Figure 6b A schematic diagram of the effect of three-dimensional reconstruction provided by another embodiment of the present disclosure;

[0025] Figure 6c A schematic diagram of the effect of three-dimensional reconstruction provided by another embodiment of the present disclosure;

[0026] Figure 7 A schematic diagram of the structure of a three-dimensional modeling device provided in one embodiment of the present disclosure;

[0027] Figure 8 A schematic diagram of the structure of an electronic device provided in one embodiment of the present disclosure. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of the present disclosure, the technical solutions in one or more embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in one or more embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on one or more embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the protection scope of the present disclosure.

[0029] The disclosed embodiment provides a 3D modeling method, which can be applied in a scenario of generating a 3D model in real time, improves the 3D modeling speed, and meets the real-time generation requirements of the 3D model.

[0030] The scene of generating a three-dimensional model in real time may be a scene of generating a volumetric video in real time and transmitting the volumetric video, or a scene of generating a virtual object in real time and broadcasting the virtual object live, wherein the scene of generating a volumetric video in real time and transmitting the volumetric video may be a scene of broadcasting the volumetric video live. For example, a user watches a volumetric video live broadcast through a terminal device, a server generates a three-dimensional model in each frame of the video in real time and sends it to the terminal device, the terminal device receives the three-dimensional model sent by the server, renders each frame of the volumetric video based on the three-dimensional model in each frame of the video, and displays the video image of each frame of the volumetric video, wherein each frame of the volumetric video includes a three-dimensional model. Among them, the user can watch the volumetric video live broadcast through a VR (Virtual Reality) device, or watch the volumetric video live broadcast through electronic devices such as mobile phones, tablet computers, car computers, wearable devices, etc.

[0031] Figure 1 A flow chart of a three-dimensional modeling method provided by an embodiment of the present disclosure, the method can be applied on a server and executed by the server, such as Figure 1 As shown, the process includes:

[0032] Step S102, obtaining the voxel space of the object; the voxel space is used to represent the physical space where the object is located;

[0033] Step S104, dividing the voxel space into multiple levels of voxel units; wherein each level includes multiple voxel units; each voxel unit includes one or more voxels; each voxel unit of the first level is obtained by dividing the voxel space; the voxel units of the next level are located inside the voxel units of the previous level, and the voxel units of the next level are obtained by dividing specific voxel units in the voxel units of the previous level, and the specific voxel units include voxels located on the surface of the three-dimensional model of the object; the voxel is used to represent the spatial area in the physical space where the object is located;

[0034] Step S106: Building a three-dimensional model of the object based on the voxel units at the last level of the division.

[0035] In this embodiment, the voxel space of the object is obtained, and the voxel space is used to represent the physical space where the object is located. The voxel space is divided into multiple levels of voxel units, wherein each level includes multiple voxel units; each voxel unit includes one or more voxels; each voxel unit of the first level is obtained by dividing the voxel space; the voxel units of the next level are located inside the voxel units of the previous level, and the voxel units of the next level are obtained by dividing specific voxel units in the voxel units of the previous level, and the specific voxel units include voxels located on the surface of the three-dimensional model of the object; voxels are used to represent the spatial area in the physical space where the object is located; based on the voxel units of the last level of division, a three-dimensional model of the object is established. It can be seen that through this embodiment, the voxel space can be divided into multiple levels of voxel units layer by layer, and the division is based on whether the voxel units of the previous level that have been divided include voxels located on the surface of the three-dimensional model of the object, and based on the voxel units of the last level of division, a three-dimensional model of the object is established. Therefore, when performing three-dimensional modeling, there is no need to pay attention to all the voxels in the voxel space, only the voxel units of the last level need to be paid attention to to achieve three-dimensional modeling, which reduces the amount of data used in the modeling process and improves the three-dimensional modeling speed in the scenario of real-time generation of three-dimensional models.

[0036] Below Figure 1 The method flow is described in detail.

[0037] In the above step S102, the voxel space of the object is obtained, and the voxel space is used to represent the physical space where the object is located. In this embodiment, the object may be an object to be modeled, and the object to be modeled includes but is not limited to people, animals, virtual objects, etc. When modeling the object to be modeled (hereinafter referred to as the object), the object is set in a physical space, and the physical space may be a space such as a studio of a specific size. Based on this, in this embodiment, the physical space where the object is located is represented by the voxel space, and the voxel space includes a plurality of voxels, each voxel corresponds to a spatial area in the physical space where the object is located, and is used to represent the spatial area. It can also be understood that the voxel space and the voxel are the physical space where the object is located and the spatial areas in the physical space are expressed in the computer system.

[0038] In the above step S104, the voxel space is divided into multiple levels of voxel units according to a preset rule. After the division, each level includes multiple voxel units, each voxel unit includes one or more voxels, and each voxel unit of the first level is obtained by dividing the voxel space, and the voxel unit of the next level is located inside the voxel unit of the previous level, and the voxel unit of the next level is obtained by dividing the specific voxel unit in the voxel unit of the previous level, and the specific voxel unit is a voxel unit including voxels located on the surface of the three-dimensional model of the object. As mentioned above, in this embodiment, the voxel is used to represent the spatial area in the physical space where the object is located.

[0039] As mentioned above, the voxel space includes a large number of voxels, and voxels are the expression of each spatial region in the physical space where the object is located in the computer system. Figure 2a This is a schematic diagram of the existing technology for dividing the voxel space, such as Figure 2a As shown, in the existing scheme, the voxel space is usually divided into voxels one by one, and each voxel is processed to establish a three-dimensional model. However, a large number of voxels need to be divided in this process, and storage space needs to be opened for each voxel, resulting in a large computer resource overhead and slow modeling speed. In this embodiment, the voxel space is divided into multiple levels of voxel units, each level includes multiple voxel units, and each voxel unit in each level includes one or more voxels, and each voxel unit of the first level is obtained by dividing the voxel space, and the voxel unit of the next level is located inside the voxel unit of the previous level, and the voxel unit of the next level is obtained by dividing specific voxel units in the voxel unit of the previous level, and the specific voxel unit includes voxels located on the surface of the three-dimensional model of the object.

[0040] In one embodiment, voxel units of each level are obtained by dividing the voxel space into multiple first-level voxel units, and the voxel units of the first level are used as voxel units to be processed. When the voxel units to be processed include voxels located on the surface of the three-dimensional model of the object, the voxel units to be processed are divided into multiple next-level voxel units, and the voxel units of the next level are used as voxel units to be processed, and the voxel units to be processed are repeatedly divided until the voxel units of the last level are divided.

[0041] In this embodiment, the number of levels of voxel units is preset. In the process of dividing the voxel space into multiple levels of voxel units, the voxel space is first divided into multiple first-level voxel units, for example, divided into 16*16*16 first-level voxel units. Then, for each first-level voxel unit, each first-level voxel unit is used as a voxel unit to be processed, and it is determined whether the interior of the voxel unit to be processed includes voxels located on the surface of the three-dimensional model of the object. If included, the voxel unit to be processed is divided into multiple second-level voxel units, for example, divided into 4*4*4 second-level voxel units. If not, it is not divided. Then, each second-level voxel unit is used as a voxel unit to be processed, and so on. The voxel unit to be processed is divided repeatedly until the voxel unit of the last level is obtained.

[0042] In this embodiment, a voxel unit of an upper layer that does not include a voxel located on the surface of the three-dimensional model of the object is not divided into a plurality of voxel units of a lower layer. Figure 2b A schematic diagram of dividing the voxel space provided by an embodiment of the present disclosure is shown in FIG. Figure 2b As shown, when dividing the voxel units of the first level, the voxel space can be divided into a plurality of voxel units of the first level. Figure 2b Three first-level voxel units are taken as an example for illustration, and these three first-level voxel units include voxels located on the surface of the three-dimensional model of the object. These three first-level voxel units need to be divided into multiple next-level voxel units.

[0043] Figure 2c A schematic diagram of dividing the voxel space provided in another embodiment of the present disclosure is as follows: Figure 2c As shown in the figure, when dividing the second-level voxel units, Figure 2b The three first-level voxel units are divided, and each of the three first-level voxel units is divided into a plurality of second-level voxel units. Figure 2c In each of the three first-level voxel units mentioned above, three second-level voxel units are schematically represented, and these three second-level voxel units include voxels located on the surface of the three-dimensional model of the object. The three second-level voxel units can be divided into multiple next-level voxel units, or, when the number of levels of voxel units is pre-set to 2 layers, it is determined that the voxel space division is completed.

[0044] Combination Figure 2a , Figure 2b and Figure 2cIt can be seen that compared with the existing solutions, in this embodiment, only the voxel units of the previous level including the voxels located on the surface of the three-dimensional model of the object are divided into multiple voxel units of the next level, which reduces the computer resource overhead used when dividing the voxel space, improves the efficiency of voxel space division, and improves the modeling speed.

[0045] After the voxel space is divided, in the above step S106, a three-dimensional model of the object is established based on the voxel units of the last level of division. This process will be described later.

[0046] In one embodiment, the above method flow further includes the following steps to determine whether the voxel unit of the previous level includes voxels located on the surface of the three-dimensional model of the object, that is, to determine whether the voxel unit of the previous level is a specific voxel unit:

[0047] Obtain the distance between the voxel unit of the previous layer and each viewing angle and the depth value of the pixel point projected by the voxel unit of the previous layer on each depth map; the depth map is the depth map of the object at each viewing angle;

[0048] According to the distance and depth values ​​corresponding to each viewing angle, it is determined whether the voxel unit of the previous level is a specific voxel unit.

[0049] In this embodiment, the voxel unit of the previous level includes multiple voxels, each voxel has a coordinate value (x, y, z) in the voxel space, and the distance between each voxel and each viewing angle is determined according to the coordinate z value of each voxel, wherein one voxel corresponds to a distance under one viewing angle, and the viewing angle refers to each viewing angle of the photographed object, and each viewing angle has a depth map of the object. For each viewing angle, the maximum distance and the minimum distance are determined from each distance under the viewing angle corresponding to each voxel included in the voxel unit of the previous level, as the maximum distance and the minimum distance between the voxel unit of the previous level and the viewing angle.

[0050] And, for each viewing angle, project the voxel unit of the previous layer onto the depth map of the viewing angle to obtain multiple pixel points obtained by projecting the voxel unit of the previous layer onto the depth map of the viewing angle, and sample the depth values ​​of the multiple pixel points to obtain the depth values ​​of the pixel points obtained by projecting the voxel unit of the previous layer onto the depth map of the viewing angle. The depth map is a depth map of the object at each viewing angle.

[0051] Finally, based on the maximum distance and minimum distance between the voxel unit of the previous level corresponding to each perspective and the perspective, as well as the depth value of the pixel point obtained by projecting the voxel unit of the previous level on the depth map of each perspective, it is determined whether the voxel unit of the previous level includes voxels located on the surface of the three-dimensional model of the object, thereby judging whether the voxel unit of the previous level is a specific voxel unit.

[0052] It can be seen that through this embodiment, the distance between the voxel unit of the previous layer and each perspective and the depth value of the pixel point projected by the voxel unit of the previous layer on each depth map can be obtained, and according to the distance and depth value corresponding to each perspective, it is possible to accurately determine whether the voxel unit of the previous layer is a specific voxel unit.

[0053] In one embodiment, determining whether a voxel unit at a previous level is a specific voxel unit according to the distance and depth value corresponding to each viewing angle includes:

[0054] If the comparison result between the distance and the depth value corresponding to any viewing angle meets the set condition, the voxel unit of the previous level is determined to be a specific voxel unit.

[0055] In this embodiment, after obtaining the maximum distance and the minimum distance between the voxel unit of the previous layer corresponding to each perspective and the perspective, as well as the depth value of the pixel point obtained by projecting the voxel unit of the previous layer on the depth map of each perspective, for any perspective, if the depth value corresponding to the perspective is between the above-mentioned maximum distance and minimum distance, it is determined that the voxel unit of the previous layer includes voxels located on the surface of the three-dimensional model of the object, and the voxel unit of the previous layer is a specific voxel unit.

[0056] Figure 3 A schematic diagram of a principle for determining whether a voxel unit at a previous level is a specific voxel unit provided by an embodiment of the present disclosure, such as Figure 3 As shown, for any viewpoint, there is a maximum distance z between the voxel unit of the previous level (i.e., the spatial node in the figure) and the viewpoint far and the minimum distance z near , and the voxel unit of the previous level is projected on the depth map of the viewing angle to obtain the depth value d of the pixel point. near <d<z far , it is determined that the voxel unit of the previous level includes the voxels located on the surface of the three-dimensional model of the object, and the voxel unit of the previous level is determined to be a specific voxel unit.

[0057] Of course, the above-described process of determining whether a voxel unit at the previous level is a specific voxel unit can be applied to any voxel unit to determine whether any voxel unit is a specific voxel unit.

[0058] It can be seen that through this embodiment, if the comparison result between the distance and depth value corresponding to any perspective meets the set conditions, it is determined that the voxel unit of the previous level includes the voxels located on the surface of the three-dimensional model of the object, which is a specific voxel unit, thereby efficiently and quickly determining whether the voxel unit of the previous level is a specific voxel unit through data comparison.

[0059] In one embodiment, building a three-dimensional model of the object based on the voxel units at the last level of the division includes:

[0060] determining a first voxel unit in the voxel units of the last level; the first voxel unit includes voxels located on the surface of the three-dimensional model of the object;

[0061] Acquire a truncated signed distance value of each first voxel included in the first voxel unit; the truncated signed distance value represents the distance between the first voxel and the surface of the three-dimensional model of the object;

[0062] According to the truncated signed distance value of each first voxel, the surface of the three-dimensional model of the object is extracted to establish the three-dimensional model of the object.

[0063] In this embodiment, a first voxel unit is determined in the voxel units of the last level, and the first voxel unit includes voxels located on the surface of the three-dimensional model of the object. The specific determination process may be:

[0064] For each voxel unit in the last level of voxel units, the distance between the voxel unit and each viewing angle and the depth value of the pixel point projected by the voxel unit on each depth map are obtained. The viewing angle refers to each viewing angle of the photographed object, and each viewing angle has a depth map of the object. For each voxel unit in the last level of voxel units, according to the distance and depth value of the voxel unit corresponding to each viewing angle, it is determined whether the voxel unit is the first voxel unit, so as to determine the first voxel unit in the last level of voxel units. Among them, for each voxel unit in the last level of voxel units, if the comparison result between the distance and depth value of the voxel unit corresponding to any viewing angle meets the set conditions, the voxel unit is determined to be the first voxel unit.

[0065] The process of determining the first voxel unit may also refer to the detailed description of determining whether the voxel unit of the previous level includes voxels located on the surface of the three-dimensional model of the object, which will not be repeated here.

[0066] Next, the truncated signed distance value of each first voxel included in the first voxel unit is obtained. The first voxel unit may include multiple first voxels. In one embodiment, after the voxel space is divided, the first voxel unit including the voxels located on the surface of the three-dimensional model of the object is determined in the voxel units of each last level, and the number of voxels included in each voxel unit of the last level is set. Then, each determined first voxel unit can also be divided into one or more voxels. For example, the number of levels of voxel units is preset to 2 layers, and the number of voxels included in each voxel unit of the last level is set to 8*8*8. Then, after the voxel units of the second level are divided, the first voxel unit including the voxels located on the surface of the three-dimensional model of the object is determined in each voxel unit of the second level, and each determined first voxel unit is divided into 8*8*8 voxels.

[0067] The truncated signed distance value represents the distance between the first voxel and the surface of the three-dimensional model of the object. The truncated signed distance value can be calculated by the truncated signed distance function (TSDF). TSDF is a method for calculating hidden potential surfaces in 3D reconstruction. TSDF divides the space into regular voxels and stores a truncated signed distance value for each voxel. The truncated signed distance value represents the distance from the voxel to the model surface. The truncated signed distance value of the voxel inside the model surface is negative, and the truncated signed distance value of the voxel outside the model surface is positive.

[0068] Finally, according to the truncated signed distance value of each first voxel, the surface of the three-dimensional model of the object is extracted to establish the three-dimensional model of the object.

[0069] It can be seen that through this embodiment, the first voxel unit can be determined in the voxel units of the last level, the truncated signed distance value of each first voxel included in the first voxel unit can be obtained, and the surface of the three-dimensional model of the object can be extracted according to the truncated signed distance value of each first voxel, and the three-dimensional model of the object can be established, so that three-dimensional reconstruction can be performed only according to the truncated signed distance value of the first voxel. Compared with the method in the prior art that needs to divide the voxel space into dense voxels and perform three-dimensional reconstruction according to the truncated signed distance value of each dense voxel, the computer resource overhead is greatly reduced and the efficiency of three-dimensional reconstruction is improved.

[0070] In one embodiment, after determining the first voxel unit in the voxel units of the last level, the method further includes:

[0071] Recording the identifiers of the voxel units at each level of the division and the subordinate relationships between the voxel units at each level of the division through a tree data structure;

[0072] A storage space is allocated for the first voxel unit, where the storage space is used to store the truncated signed distance value of each first voxel included in the first voxel unit.

[0073] In this embodiment, after the voxel units of each level are divided, the identifiers of the voxel units of each level and the subordinate relationship between the voxel units of each level can be recorded through the tree data structure in the memory. In the tree data structure, the voxel units of the previous level are represented by the parent node, the voxel units of the next level are represented by the child node, and the subordinate relationship between the voxel units of the previous level and the voxel units of the next level is represented by the connection relationship between the parent node and the child node.

[0074] In this embodiment, after the first voxel unit is determined in the voxel units of the last level, storage space is allocated for the first voxel unit in the GPU (Graphics Processing Unit) memory, and the truncated signed distance value of each first voxel included in the first voxel unit is stored in the storage space.

[0075] Figure 4 A schematic diagram of voxel space division and data storage provided by an embodiment of the present disclosure, such as Figure 4 As shown, the number of levels of voxel units is preset to 2 layers, and the number of voxels included in each voxel unit of the last level is set to 8*8*8. First, the voxel space is divided into 16*16*16 voxel units of the first level, and the identifier of each voxel unit of the first level is recorded. Then, the voxel units including the voxels located on the surface of the three-dimensional model of the object are determined among all the voxel units of the first level. The determined voxel units are represented by slashes in the figure. A list of determined voxel units is generated and stored, and the identifiers of the determined voxel units are recorded in the list. Moreover, each determined voxel unit is divided into 4*4*4 voxel units of the second level, and the identifier of each voxel unit of the second level and the subordinate relationship between the voxel unit of the first level and the voxel unit of the second level are recorded. The subordinate relationship can be represented by the inclusion relationship between the identifiers of the voxel units.

[0076] Next, a voxel unit including a voxel located on the surface of the three-dimensional model of the object is determined as a first voxel unit among all the second-level voxel units, and the first voxel unit is represented by a slash in the figure, and a list of the first voxel units is generated and stored, and the identifier of the first voxel unit is recorded in the list, and a storage space is opened for the first voxel unit, and the truncated signed distance value of each first voxel included in the first voxel unit is stored in the storage space for three-dimensional reconstruction. Each first voxel unit includes 8*8*8 voxels.

[0077] By generating and storing a list of first voxel units, it is convenient for a computer thread to read the list to determine the voxel units that need to be traversed when performing three-dimensional reconstruction. In one example, a continuous storage space can be opened up for the first voxel units to store the truncated signed distance value of each first voxel included in each first voxel unit. The efficiency of reading the truncated signed distance value of the first voxel from the memory is improved through the continuous storage space, thereby improving the efficiency of three-dimensional reconstruction.

[0078] It can be seen that in this embodiment, the identification of the voxel units of each divided level and the subordinate relationship between the voxel units of each divided level are recorded through a tree-like data structure, and the tree-like data structure only occupies a relatively small memory size, thereby reducing the resource consumption of the computer; by allocating storage space to store the truncated signed distance value of each first voxel included in the first voxel unit for three-dimensional reconstruction, compared with the method in the prior art that needs to store the truncated signed distance value of each dense voxel for three-dimensional reconstruction, the storage space occupied is greatly reduced, thereby reducing the resource overhead of the computer.

[0079] In one embodiment, obtaining the truncated signed distance value of each first voxel included in the first voxel unit includes:

[0080] For each first voxel, obtaining a sub-truncated signed distance value of the first voxel at each viewing angle; wherein the sub-truncated signed distance value represents a distance between the first voxel and a surface of the three-dimensional model of the object at the viewing angle;

[0081] For each first voxel, the sub-truncated signed distance values ​​of the first voxel at each viewing angle are fused to obtain the truncated signed distance value of the first voxel.

[0082] In this embodiment, for each first voxel, a sub-truncated signed distance value of the first voxel at each viewing angle is obtained, where the viewing angle refers to each viewing angle of the photographed object, each viewing angle has a depth map of the object, and the sub-truncated signed distance value represents the distance between the first voxel and the surface of the three-dimensional model of the object at each viewing angle.

[0083] Next, for each first voxel, the sub-truncated signed distance values ​​of the first voxel at each viewing angle are fused to obtain the truncated signed distance value of the first voxel. The truncated signed distance value represents the distance between the first voxel and the surface of the three-dimensional model of the object, and the distance is independent of the viewing angle. Through the truncated signed distance value, the first voxel located on the model surface can be determined, thereby performing three-dimensional reconstruction.

[0084] It can be seen that, through this embodiment, the truncated signed distance value of the first voxel can be obtained by fusing the sub-truncated signed distance values ​​of the first voxel at various viewing angles, and the truncated signed distance value of the first voxel can be quickly determined.

[0085] In one embodiment, after the truncated signed distance value of each first voxel included in the first voxel unit is obtained, the surface of the three-dimensional model of the object can be extracted by surface reconstruction, such as the Marching Cubes surface extraction algorithm, to establish the three-dimensional model of the object. For example, after the truncated signed distance value of each first voxel included in the first voxel unit is stored in a continuous storage space, a thread is allocated to each first voxel, and the thread is used to traverse the truncated signed distance value of each first voxel in each first voxel unit according to the above-mentioned list of first voxel units, and the surface of the model is reconstructed according to the truncated signed distance value of the traversed first voxel and the truncated signed distance value of its adjacent voxel in the voxel space.

[0086] Considering that when the truncated signed distance value of each first voxel included in each first voxel unit is stored in a continuous storage space, two adjacent first voxel units in the storage space, although the boundary voxels included in these two first voxel units are adjacent in the storage space, are not necessarily adjacent in the voxel space, and when determining the model surface based on the boundary voxels in the first voxel unit, it is necessary to obtain the truncated signed distance value of the voxel adjacent to the boundary voxel in the voxel space. Therefore, in order to improve the speed of obtaining the truncated signed distance value of the voxel adjacent to the boundary voxel in the voxel space from the storage space, in one embodiment, according to the truncated signed distance value of each first voxel, the surface of the three-dimensional model of the object is extracted, and the three-dimensional model of the object is established, including:

[0087] determining a first voxel located at a boundary in a first voxel unit, and determining a neighboring voxel of the first voxel located at the boundary in a voxel space;

[0088] The truncated signed distance value of the first voxel located at the boundary and the truncated signed distance values ​​of the adjacent voxels are stored in an adjacent storage space, so that the truncated signed distance value of the first voxel located at the boundary and the truncated signed distance value of the adjacent voxel can be read from the adjacent storage space, and the surface of the three-dimensional model of the object can be extracted based on the first voxel located at the boundary using the read data.

[0089] In this embodiment, for each first voxel unit, a first voxel located at the boundary is determined in the first voxel unit, and adjacent voxels of the first voxel located at the boundary are determined in the voxel space. Then, the truncated signed distance value of the first voxel located at the boundary and the determined truncated signed distance values ​​of the adjacent voxels are stored in an adjacent storage space, so that the truncated signed distance value of the first voxel located at the boundary and the truncated signed distance value of the adjacent voxels can be read from the adjacent storage space. The truncated signed distance value of the first voxel located at the boundary and the truncated signed distance value of the adjacent voxels can be used to efficiently and quickly determine the model surface based on the boundary voxels in the first voxel unit.

[0090] The above process can also be called voxel expansion. Figure 5 A schematic diagram of voxel expansion provided by an embodiment of the present disclosure is shown in FIG. Figure 5 As shown, assuming that the first voxel unit includes 8*8*8 voxels, the first voxel located at the boundary is determined in the first voxel unit, and the adjacent voxels of the first voxel located at the boundary are determined in the voxel space, and the first voxel unit can be expanded using the adjacent voxels, so that the first voxel unit becomes to include 10*10*10 voxels, and the adjacent voxels are illustrated in dark gray in the figure, and, in the storage space opened up for the first voxel unit, based on the expanded first voxel unit, the truncated signed distance value of the first voxel located at the boundary and the truncated signed distance value of the determined adjacent voxel are stored adjacently, so that the computer thread can read the truncated signed distance value of the first voxel located at the boundary and the truncated signed distance value of the adjacent voxel from the adjacent storage address, and use the read data to efficiently and quickly determine the model surface based on the boundary voxels in the first voxel unit.

[0091] It can be seen that through this embodiment, the truncated signed distance value of the first voxel located at the boundary and the truncated signed distance values ​​of the adjacent voxels of the first voxel in the voxel space can be stored in the adjacent storage space, so that the computer thread can quickly read the truncated signed distance value of the first voxel located at the boundary and the truncated signed distance value of the adjacent voxel from the adjacent storage space, and use the read data to extract the surface of the three-dimensional model of the object based on the first voxel located at the boundary, thereby improving the data reading speed and modeling speed.

[0092] In one embodiment, taking each first voxel as an example, for each first voxel, obtaining the sub-truncation signed distance value of the first voxel at each viewing angle includes:

[0093] Obtaining the distance between the first voxel and each viewing angle and the depth value of the pixel point projected by the first voxel on each depth map; the depth map is a depth map of the object at each viewing angle;

[0094] According to the distance and depth value corresponding to each viewing angle, a sub-truncation signed distance value of the first voxel at each viewing angle is determined.

[0095] In this embodiment, the first voxel has a coordinate value (x, y, z) in the voxel space, and the distance between the first voxel and each view is determined according to the coordinate z value of the first voxel, wherein the first voxel corresponds to a distance under a view. Next, the first voxel is projected onto each depth map, which is a depth map of the object at each view, and the depth value of the pixel point projected onto each depth map is obtained. Next, for each view, the sub-truncated signed distance value of the first voxel at the view is determined according to the distance between the view and the first voxel and the depth value of the pixel point obtained by projecting the first voxel onto the depth map of the view.

[0096] In one embodiment, for each viewing angle, the sub-truncated signed distance value of the first voxel at the viewing angle can be determined by the following formula (1) according to the distance between the viewing angle and the first voxel and the depth value of the pixel point obtained by projecting the first voxel onto the depth map of the viewing angle.

[0097] tsdf(p)=clamp((dp.z) / d trunc ,-1,1) (1)

[0098] In formula (1), p represents the first voxel, d represents the depth value of the pixel obtained by projecting the first voxel onto the depth map of the viewing angle, pz represents the distance between the viewing angle and the first voxel, and d trunc represents the preset truncation distance, and the sub-truncation signed distance value tsdf(p) of the first voxel at each viewing angle calculated by formula (1) has a value range of [-1, 1].

[0099] It can be seen that through this embodiment, the distance between the first voxel and each viewpoint and the depth value of the pixel point projected by the first voxel on the depth map of each viewpoint can be obtained, and the sub-truncated signed distance value of the first voxel at each viewpoint can be accurately determined according to the distance and depth value corresponding to each viewpoint, thereby improving the efficiency of determining the sub-truncated signed distance value of the first voxel at each viewpoint.

[0100] In one embodiment, for each first voxel, the sub-truncated signed distance values ​​of the first voxel at each viewing angle are fused to obtain the truncated signed distance value of the first voxel, including:

[0101] Obtain the confidence of the depth map of the object at each viewing angle; the confidence indicates the credibility of the depth value in the depth map;

[0102] The confidence of the depth map of the object at each viewing angle is used as the fusion weight of each viewing angle, and the sub-truncated signed distance values ​​of the first voxel at each viewing angle are fused to obtain the truncated signed distance value of the first voxel.

[0103] In this embodiment, the confidence of the depth map of the object at each viewing angle is first obtained. The object has a depth map at each viewing angle, and the confidence of the depth map of each viewing angle can be obtained by calculation. The confidence is used to indicate the degree of credibility of the depth value in the depth map. In one embodiment, the confidence can be indicated by a confidence map, where one depth map corresponds to a confidence map with the same resolution, and the pixel value in the confidence map indicates the degree of credibility of the depth value of the corresponding pixel point in the depth map.

[0104] Next, for each view, the confidence of the depth map of the view is used as the fusion weight of the view. For each first voxel, after the sub-truncated signed distance values ​​of the first voxel at each view are determined as above, the sub-truncated signed distance values ​​of the first voxel at each view are fused according to the fusion weight of each view to obtain the truncated signed distance value of the first voxel, which represents the distance between the first voxel and the surface of the three-dimensional model of the object.

[0105] In one embodiment, for each first voxel, the sub-truncated signed distance values ​​of the first voxel at each view angle are fused according to the fusion weight of each view angle through the following formulas (2)-(4) to obtain the truncated signed distance value of the first voxel.

[0106]

[0107] W(p)=W(p)+w(p) (3)

[0108] w(p)=1-e i (u,v) (4)

[0109] In formulas (2)-(4), p represents the first voxel, TSDF(P) is the truncated signed distance value of the first voxel obtained by fusion, tsdf(p) represents the sub-truncated signed distance value of the first voxel at the current view, w(p) is the fusion weight of the current view, W(p) is the cumulative weight of p, and e i (u, v) is the error of the depth map of the current view i.

[0110] The principle of formulas (2)-(4) is to determine the error of the depth map of each viewing angle as e i (u, v), based on this error, determine the confidence level 1-e for each view i (u, v), and the confidence is used as the fusion weight w(p) of each view. When fusion is performed, taking the first view as an example, w(p) is the fusion weight of the first view, tsdf(p) is the sub-truncated signed distance value of the first voxel in the first view, and W(p) and TSDF(P) have preset initial values.

[0111] When fusing the second view, TSDF(P) is the truncated signed distance value of the first voxel determined when the fusion of the first view is completed, W(p) is the cumulative weight determined when the fusion of the first view is completed, w(p) is the fusion weight of the second view, and tsdf(p) is the sub-truncated signed distance value of the first voxel in the second view.

[0112] When fusing the third view, TSDF(P) is the truncated signed distance value of the first voxel determined when the fusion of the second view is completed, W(p) is the cumulative weight determined when the fusion of the second view is completed, w(p) is the fusion weight of the third view, and tsdf(p) is the sub-truncated signed distance value of the first voxel in the third view.

[0113] It can be seen that through this embodiment, the confidence of the depth map of the object at each perspective can be obtained as the fusion weight of each perspective, and for each first voxel, the sub-truncated signed distance values ​​of the first voxel at each perspective are fused according to the fusion weights of each perspective, so as to quickly and accurately determine the truncated signed distance value of the first voxel.

[0114] In one embodiment, a thread is allocated to each first voxel, and the truncated signed distance value of each first voxel is calculated by the thread, and the truncated signed distance value of each first voxel is recorded in the tree data structure.

[0115] In one embodiment, obtaining the confidence of the depth map of the object at each viewing angle includes:

[0116] Obtain a depth map of the object at each viewing angle, a normal map of the object at each viewing angle, a direction vector of each viewing angle, and a viewing angle parameter of each viewing angle;

[0117] The confidence of the depth map of the object at each viewing angle is determined according to the depth map of the object at each viewing angle, the normal map of the object at each viewing angle, the orientation direction vector of each viewing angle and the viewing angle parameter of each viewing angle.

[0118] In this embodiment, taking any one viewing angle as an example, a depth map of the object processed at the viewing angle, a normal map of the object at the viewing angle, a direction vector of the viewing angle, and a viewing angle parameter of the viewing angle are obtained. According to the depth map of the object at the viewing angle, the normal map of the object at the viewing angle, the direction vector of the viewing angle, and the viewing angle parameter of the viewing angle, the error of the depth map of the object at the viewing angle is determined as e i (u, v), calculate 1-e i (u, v) is used as the confidence of the viewing angle. i (u, v) may be an error map with the same resolution as the depth map, and the pixel values ​​of the pixels in the error map represent the error values ​​of the depth values ​​of the corresponding pixels in the depth map.

[0119] In one example, taking a viewing angle as an example, the error map e of the depth map of the viewing angle is calculated by the following formula (5): i (u, v).

[0120]

[0121] In formula (5), (u, v) represents any pixel in the depth map, i represents the current viewing angle, and e i (u, v) represents the error map of the depth map, Q is the normalization coefficient, D i is the depth map after denoising of the perspective. The denoising method can be bilateral filtering, local window depth range, etc. i is the normal map of the object at this viewing angle. The normal map can be generated based on the depth map of this viewing angle, or based on the RGB map of this viewing angle. i is the direction vector of the view in the world coordinate system, σ is a preset parameter, B represents the boundary of the object in the denoised depth map, U represents the unsigned distance field determined based on the boundary of the object in the denoised depth map, and the unsigned distance field records the distance from each pixel to the nearest boundary, f i Indicates the focal length of the camera at this viewing angle, that is, the viewing angle parameter of this viewing angle.

[0122] The principle of formula (5) is that, for any viewing angle, the boundary of the object in the depth map of the viewing angle is obtained, and the unsigned distance field of the viewing angle is determined based on the boundary of the object in the depth map. The unsigned distance field records the distance of each pixel from the nearest boundary. Based on the boundary of the viewing angle, the unsigned distance field, the depth map of the object at the viewing angle, the normal map of the object at the viewing angle, the heading direction vector of the viewing angle and the viewing parameters of the viewing angle, the error map of the depth map of the viewing angle is generated by the above formula (5).

[0123] It can be seen that through this embodiment, the confidence of the depth map of the object at each viewing angle can be accurately determined based on the depth map of the object at each viewing angle, the normal map of the object at each viewing angle, the orientation direction vector of each viewing angle and the viewing angle parameters of each viewing angle.

[0124] Figure 6a A schematic diagram of the effect of three-dimensional reconstruction provided by an embodiment of the present disclosure is shown in FIG. Figure 6a As shown, multiple first-level voxel units are obtained by dividing in the voxel space. Figure 6a Specific voxel units including voxels located on the surface of the model determined from a plurality of first-level voxel units are illustrated in FIG. It can be seen that the specific voxel units of the first level are all located near the object, which is illustrated as a person in the figure.

[0125] Figure 6b A schematic diagram of the effect of three-dimensional reconstruction provided by another embodiment of the present disclosure is shown as follows: Figure 6b As shown, a plurality of voxel units of the second level are obtained by dividing a specific voxel unit of the first level. Figure 6b , which illustrates a specific voxel unit including a voxel located on the surface of the model determined from a plurality of specific voxel units of the second level, Figure 6b In FIG. 1 , each specific voxel unit of the second level includes 8*8*8 voxels. It can be seen that the specific voxel units of the second level are all located near the object, which is illustrated as a person in the figure.

[0126] Figure 6c A schematic diagram of the effect of three-dimensional reconstruction provided by another embodiment of the present disclosure is shown as follows: Figure 6c As shown, Figure 6a and Figure 6b Superposition, we get Figure 6c ,from Figure 6c It can be seen that compared with the prior art, the present embodiment requires fewer voxel units to be divided, thereby saving computer resources, improving modeling speed, and reducing video memory usage.

[0127] In summary, a three-dimensional modeling method is provided. Compared with the existing solution of dividing the voxel space into multiple dense voxels and assigning a thread to each dense voxel to calculate the truncated signed distance value for three-dimensional reconstruction, the three-dimensional reconstruction method in the above embodiment can divide the voxel space into multiple levels of voxel units layer by layer, and the division is based on whether the voxel units of the previous level that have been divided include voxels located on the surface of the three-dimensional model of the object, and threads are assigned to the voxels in the voxel units of the last level of division to establish a three-dimensional model of the object. Therefore, when performing three-dimensional modeling, there is no need to pay attention to all the voxels in the voxel space, and only the sparse spatial voxels are assigned to threads for TSDF fusion and surface extraction, which greatly reduces the relevant calculations of voxels in areas without objects in the space and their corresponding video memory occupancy, reduces the amount of data used in the modeling process, and improves the three-dimensional modeling speed in the scene of real-time generation of three-dimensional models.

[0128] Moreover, through the above three-dimensional reconstruction method, since the first voxel unit is determined in the voxel units of the last level, the three-dimensional model is reconstructed using the first voxel unit, and the first voxel unit includes voxels located on the surface of the model, the above method will not reduce the resolution of the reconstructed three-dimensional model, and the modeling speed is improved while ensuring the model resolution.

[0129] An embodiment of the present disclosure provides a three-dimensional modeling device, which is used to implement the above three-dimensional modeling method. Figure 7 A schematic diagram of the structure of a three-dimensional modeling device provided in an embodiment of the present disclosure, such as Figure 7 As shown, the device comprises:

[0130] An acquisition unit 71 is used to acquire a voxel space of an object; the voxel space is used to represent a physical space where the object is located;

[0131] A division unit 72, used for dividing the voxel space into a plurality of levels of voxel units; wherein each level includes a plurality of voxel units; each of the voxel units includes one or more voxels; each voxel unit of the first level is obtained by dividing the voxel space; the voxel units of the next level are located inside the voxel units of the previous level, and the voxel units of the next level are obtained by dividing specific voxel units in the voxel units of the previous level, and the specific voxel units include voxels located on the surface of the three-dimensional model of the object; the voxels are used to represent the spatial area in the physical space where the object is located;

[0132] The modeling unit 73 is used to establish a three-dimensional model of the object based on the voxel units at the last level of the division.

[0133] Optionally, the above device further includes a judging unit, configured to:

[0134] Acquire the distance between the voxel unit of the previous layer and each viewing angle and the depth value of the pixel point projected by the voxel unit of the previous layer on each depth map; the depth map is the depth map of the object at each viewing angle;

[0135] According to the distance and the depth value corresponding to each of the viewing angles, it is determined whether the voxel unit at the previous level is the specific voxel unit.

[0136] Optionally, the judging unit is specifically configured to:

[0137] If the comparison result between the distance corresponding to any one of the viewing angles and the depth value meets a set condition, the voxel unit at the previous level is determined to be the specific voxel unit.

[0138] Optionally, the modeling unit is specifically used for:

[0139] Determining a first voxel unit among the voxel units of the last level; the first voxel unit includes voxels located on the surface of the three-dimensional model of the object;

[0140] Acquire a truncated signed distance value of each first voxel included in the first voxel unit; the truncated signed distance value represents the distance between the first voxel and the surface of the three-dimensional model of the object;

[0141] According to the truncated signed distance value of each of the first voxels, the surface of the three-dimensional model of the object is extracted to establish the three-dimensional model of the object.

[0142] Optionally, a storage unit is further included, for:

[0143] After determining the first voxel unit in the voxel units of the last level, recording the identifiers of the voxel units of each level of the division and the subordinate relationships between the voxel units of each level of the division through a tree data structure;

[0144] A storage space is allocated for the first voxel unit, where the storage space is used to store the truncated signed distance value of each first voxel included in the first voxel unit.

[0145] Optionally, the modeling unit is further specifically used for:

[0146] For each of the first voxels, obtaining a sub-truncated signed distance value of the first voxel at each viewing angle; wherein the sub-truncated signed distance value represents a distance between the first voxel and a surface of the three-dimensional model of the object at the viewing angle;

[0147] For each of the first voxels, the sub-truncated signed distance values ​​of the first voxel at each of the viewing angles are fused to obtain the truncated signed distance value of the first voxel.

[0148] Optionally, the modeling unit is further specifically used for:

[0149] Determine the first voxel located at the boundary in the first voxel unit, and determine the neighboring voxels of the first voxel located at the boundary in the voxel space;

[0150] The truncated signed distance value of the first voxel located at the boundary and the truncated signed distance value of the adjacent voxel are stored in an adjacent storage space, so that the truncated signed distance value of the first voxel located at the boundary and the truncated signed distance value of the adjacent voxel are read from the adjacent storage space, and the surface of the three-dimensional model of the object is extracted based on the first voxel located at the boundary using the read data.

[0151] Optionally, the modeling unit is further specifically used for:

[0152] Acquire the distance between the first voxel and each of the viewing angles and the depth value of the pixel point projected by the first voxel on each depth map; the depth map is a depth map of the object at each of the viewing angles;

[0153] A sub-truncated signed distance value of the first voxel at each of the viewing angles is determined according to the distance and the depth value corresponding to each of the viewing angles.

[0154] Optionally, the modeling unit is further specifically used for:

[0155] Acquire the confidence of the depth map of the object at each of the viewing angles; the confidence indicates the credibility of the depth value in the depth map;

[0156] The confidence of the depth map of the object at each viewing angle is used as a fusion weight of each viewing angle, and the sub-truncated signed distance values ​​of the first voxel at each viewing angle are fused to obtain the truncated signed distance value of the first voxel.

[0157] Optionally, the modeling unit is further specifically used for:

[0158] Acquire a depth map of the object at each of the viewing angles, a normal map of the object at each of the viewing angles, a direction vector of each of the viewing angles, and a viewing angle parameter of each of the viewing angles;

[0159] The confidence of the depth map of the object at each viewing angle is determined according to the depth map of the object at each viewing angle, the normal map of the object at each viewing angle, the orientation direction vector of each viewing angle and the viewing angle parameter of each viewing angle.

[0160] The three-dimensional modeling device in the embodiment of the present disclosure can implement each process of the above-mentioned three-dimensional modeling method embodiment and achieve the same effects and functions, which will not be repeated here.

[0161] An embodiment of the present disclosure further provides an electronic device, Figure 8 A schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure, such as Figure 8 As shown, the electronic device may have relatively large differences due to different configurations or performances, and may include one or more processors 801 and memory 802, and one or more applications or data may be stored in the memory 802. Among them, the memory 802 may be a temporary storage or a permanent storage. The application stored in the memory 802 may include one or more modules (not shown in the figure), and each module may include a series of computer executable instructions in the electronic device. Furthermore, the processor 801 can be configured to communicate with the memory 802 to execute a series of computer executable instructions in the memory 802 on the electronic device. The electronic device may also include one or more power supplies 803, one or more wired or wireless network interfaces 804, one or more input or output interfaces 805, one or more keyboards 806, etc.

[0162] In a specific embodiment, the electronic device is a background server, including a processor; and a memory configured to store computer executable instructions, wherein when the computer executable instructions are executed, the processor implements the following process:

[0163] Acquire a voxel space of an object; the voxel space is used to represent a physical space where the object is located;

[0164] The voxel space is divided into a plurality of levels of voxel units; wherein each level includes a plurality of voxel units; each of the voxel units includes one or more voxels; each voxel unit of the first level is obtained by dividing the voxel space; the voxel units of the next level are located inside the voxel units of the previous level, and the voxel units of the next level are obtained by dividing specific voxel units in the voxel units of the previous level, and the specific voxel units include voxels located on the surface of the three-dimensional model of the object; the voxels are used to represent the spatial area in the physical space where the object is located;

[0165] A three-dimensional model of the object is established based on the voxel units at the last level of the division.

[0166] The electronic device in the embodiment of the present disclosure can implement each process of the above-mentioned three-dimensional modeling method embodiment and achieve the same effects and functions, which will not be repeated here.

[0167] Another embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the following process is implemented:

[0168] Acquire a voxel space of an object; the voxel space is used to represent a physical space where the object is located;

[0169] The voxel space is divided into a plurality of levels of voxel units; wherein each level includes a plurality of voxel units; each of the voxel units includes one or more voxels; each voxel unit of the first level is obtained by dividing the voxel space; the voxel units of the next level are located inside the voxel units of the previous level, and the voxel units of the next level are obtained by dividing specific voxel units in the voxel units of the previous level, and the specific voxel units include voxels located on the surface of the three-dimensional model of the object; the voxels are used to represent the spatial area in the physical space where the object is located;

[0170] A three-dimensional model of the object is established based on the voxel units at the last level of the division.

[0171] The storage medium in the embodiment of the present disclosure can implement each process of the above-mentioned three-dimensional modeling method embodiment and achieve the same effects and functions, which will not be repeated here.

[0172] In various embodiments of the present disclosure, the computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0173] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0174] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0175] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0176] For the convenience of description, the above devices are described in terms of functions and are divided into various units. Of course, when implementing the embodiments of the present disclosure, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0177] Those skilled in the art will appreciate that one or more embodiments of the present disclosure may be provided as a method, system or computer program product. Therefore, one or more embodiments of the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, one or more embodiments of the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0178] The present disclosure is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present disclosure. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0179] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0180] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0181] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0182] One or more embodiments of the present disclosure may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of the present disclosure may also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0183] Each embodiment in the present disclosure is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0184] The above description is only an embodiment of the present disclosure and is not intended to limit the present disclosure. For those skilled in the art, the present disclosure may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present disclosure shall be included in the scope of the claims of the present disclosure.

Claims

1. A three-dimensional modeling method, characterized in that: include: Acquire a voxel space of an object; the voxel space is used to represent a physical space where the object is located; The voxel space is divided into a plurality of levels of voxel units; wherein each level includes a plurality of voxel units; each of the voxel units includes one or more voxels; each voxel unit of the first level is obtained by dividing the voxel space; the voxel units of the next level are located inside the voxel units of the previous level, and the voxel units of the next level are obtained by dividing specific voxel units in the voxel units of the previous level, and the specific voxel units include voxels located on the surface of the three-dimensional model of the object; the voxels are used to represent the spatial area in the physical space where the object is located; A three-dimensional model of the object is established based on the voxel units at the last level of the division.

2. The method according to claim 1, characterized in that The method further comprises: Acquire the distance between the voxel unit of the previous layer and each viewing angle and the depth value of the pixel point projected by the voxel unit of the previous layer on each depth map; the depth map is the depth map of the object at each viewing angle; According to the distance and the depth value corresponding to each of the viewing angles, it is determined whether the voxel unit at the previous level is the specific voxel unit.

3. The method according to claim 2, characterized in that The determining, according to the distance and the depth value corresponding to each of the viewing angles, whether the voxel unit at the previous level is the specific voxel unit comprises: If the comparison result between the distance corresponding to any one of the viewing angles and the depth value meets a set condition, the voxel unit at the previous level is determined to be the specific voxel unit.

4. The method according to claim 1, characterized in that: The step of establishing a three-dimensional model of the object based on the voxel units at the last level of the division includes: Determining a first voxel unit among the voxel units of the last level; the first voxel unit includes voxels located on the surface of the three-dimensional model of the object; Acquire a truncated signed distance value of each first voxel included in the first voxel unit; the truncated signed distance value represents the distance between the first voxel and the surface of the three-dimensional model of the object; According to the truncated signed distance value of each of the first voxels, the surface of the three-dimensional model of the object is extracted to establish the three-dimensional model of the object.

5. The method according to claim 4, characterized in that After determining the first voxel unit in the voxel units of the last level, the method further includes: Recording the identifiers of the voxel units at each level of the division and the subordinate relationships between the voxel units at each level of the division through a tree data structure; A storage space is allocated for the first voxel unit, where the storage space is used to store the truncated signed distance value of each first voxel included in the first voxel unit.

6. The method according to claim 4, characterized in that The obtaining of the truncated signed distance value of each first voxel included in the first voxel unit includes: For each of the first voxels, obtaining a sub-truncated signed distance value of the first voxel at each viewing angle; wherein the sub-truncated signed distance value represents a distance between the first voxel and a surface of the three-dimensional model of the object at the viewing angle; For each of the first voxels, the sub-truncated signed distance values ​​of the first voxel at each of the viewing angles are fused to obtain the truncated signed distance value of the first voxel.

7. The method according to claim 4, characterized in that The step of extracting the surface of the three-dimensional model of the object according to the truncated signed distance value of each of the first voxels and establishing the three-dimensional model of the object comprises: Determine the first voxel located at the boundary in the first voxel unit, and determine the neighboring voxels of the first voxel located at the boundary in the voxel space; The truncated signed distance value of the first voxel located at the boundary and the truncated signed distance value of the adjacent voxel are stored in an adjacent storage space, so that the truncated signed distance value of the first voxel located at the boundary and the truncated signed distance value of the adjacent voxel are read from the adjacent storage space, and the surface of the three-dimensional model of the object is extracted based on the first voxel located at the boundary using the read data.

8. The method according to claim 6, characterized in that The obtaining of the sub-truncation signed distance value of the first voxel at each viewing angle includes: Acquire the distance between the first voxel and each of the viewing angles and the depth value of the pixel point projected by the first voxel on each depth map; the depth map is a depth map of the object at each of the viewing angles; A sub-truncated signed distance value of the first voxel at each of the viewing angles is determined according to the distance and the depth value corresponding to each of the viewing angles.

9. The method according to claim 6, characterized in that The step of fusing the sub-truncated signed distance values ​​of the first voxel at each of the viewing angles to obtain the truncated signed distance value of the first voxel includes: Acquire the confidence of the depth map of the object at each of the viewing angles; the confidence indicates the credibility of the depth value in the depth map; The confidence of the depth map of the object at each viewing angle is used as a fusion weight of each viewing angle, and the sub-truncated signed distance values ​​of the first voxel at each viewing angle are fused to obtain the truncated signed distance value of the first voxel.

10. The method according to claim 9, characterized in that The obtaining the confidence of the depth map of the object at each viewing angle includes: Acquire a depth map of the object at each of the viewing angles, a normal map of the object at each of the viewing angles, a direction vector of each of the viewing angles, and a viewing angle parameter of each of the viewing angles; The confidence of the depth map of the object at each viewing angle is determined according to the depth map of the object at each viewing angle, the normal map of the object at each viewing angle, the orientation direction vector of each viewing angle and the viewing angle parameter of each viewing angle.

11. A three-dimensional modeling device, characterized in that: include: An acquisition unit, used for acquiring a voxel space of an object; The voxel space is used to represent the physical space where the object is located; A division unit, used for dividing the voxel space into a plurality of levels of voxel units; wherein each level includes a plurality of voxel units; each of the voxel units includes one or more voxels; each voxel unit of the first level is obtained by dividing the voxel space; the voxel units of the next level are located inside the voxel units of the previous level, and the voxel units of the next level are obtained by dividing specific voxel units in the voxel units of the previous level, and the specific voxel units include voxels located on the surface of the three-dimensional model of the object; the voxels are used to represent the spatial area in the physical space where the object is located; The modeling unit is used to establish a three-dimensional model of the object based on the voxel units at the last level of the division.

12. An electronic device, characterized in that: include: processor; as well as, A memory configured to store computer executable instructions, which, when executed, cause the processor to implement the steps of the method according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the steps of the method described in any one of claims 1 to 10.

Images