Three-dimensional model reconstruction method and device, and electronic device

Abstract

The embodiment of the present disclosure relates to a three-dimensional model reconstruction method and device, and electronic equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: obtaining a depth map of a to-be-processed image, and performing instance segmentation on the to-be-processed image to obtain an instance segmentation map; performing voxel reconstruction on a target object in the to-be-processed image based on the instance segmentation map, and obtaining a voxel reconstruction model; fitting and calculating voxel features corresponding to the voxel reconstruction model, image features corresponding to the to-be-processed image, and global depth features corresponding to the depth map, obtaining an occupancy value of a three-dimensional point for implicit reconstruction; and performing three-dimensional reconstruction on the target object in the to-be-processed image based on the occupancy value of the three-dimensional point, and obtaining a three-dimensional human body model of the target object. The present disclosure can improve the accuracy of the three-dimensional human body model reconstructed in a shielding scene.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, and more specifically, to a three-dimensional model reconstruction method, a three-dimensional model reconstruction device, and an electronic device. Background Technology

[0002] In the process of constructing a realistic 3D human body model in the meta-world to build a virtual digital human, there may be problems of people occluding each other in scenes with many people gathered together.

[0003] In related technologies, parametric reconstruction approaches predict the parameters of a human body template and then deform the base object model to achieve 3D reconstruction of the target object. However, the object template method only models the object itself, which has limitations and results in a less accurate model, making comprehensive and precise reconstruction difficult.

[0004] It should be noted that the information in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this disclosure is to provide a three-dimensional model reconstruction method, a three-dimensional model reconstruction device, and an electronic device, thereby overcoming, to at least a certain extent, the problem of low accuracy of reconstructed three-dimensional human body models caused by the limitations and defects of related technologies.

[0006] According to one aspect of this disclosure, a three-dimensional model reconstruction method is provided, comprising: acquiring a depth map of an image to be processed, and performing instance segmentation on the image to be processed to obtain an instance segmentation map; performing voxel reconstruction on a target object in the image to be processed based on the instance segmentation map to obtain a voxel reconstruction model; performing fitting calculations on voxel features corresponding to the voxel reconstruction model, image features corresponding to the image to be processed, and global depth features corresponding to the depth map to obtain occupancy values ​​of three-dimensional points for implicit reconstruction; and performing three-dimensional reconstruction on the target object in the image to be processed based on the occupancy values ​​of the three-dimensional points to obtain a three-dimensional human model of the target object.

[0007] In one exemplary embodiment of this disclosure, the step of performing voxel reconstruction on the target object in the image to be processed based on the instance segmentation map to obtain a voxel reconstruction model includes: performing voxel reconstruction on the instance corresponding to the target object in the instance segmentation map through a voxel estimation network to obtain the voxel reconstruction model; wherein the voxel estimation network is trained based on 3D reconstruction loss and contour occlusion loss.

[0008] In an exemplary embodiment of this disclosure, the step of performing voxel reconstruction on the instance corresponding to the target object in the instance segmentation map through a voxel estimation network to obtain the voxel reconstruction model includes: determining whether each voxel of the instance corresponding to the target object is located in the three-dimensional object model through the voxel estimation network to determine the existence state; if the existence state is that the voxel is located in the three-dimensional object model, then constructing the voxel reconstruction model based on the voxel.

[0009] In one exemplary embodiment of this disclosure, the step of fitting and calculating the voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map to obtain the occupancy value of the three-dimensional point for implicit reconstruction includes: fusing the voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map to obtain a first hybrid feature; predicting the first hybrid feature based on the implicit function to obtain the occupancy value of the three-dimensional point, so as to determine whether the three-dimensional point is within the target grid.

[0010] In one exemplary embodiment of this disclosure, the method further includes: monitoring the predicted occupancy value and the actual occupancy value based on the difference between the predicted occupancy value and the actual occupancy value.

[0011] In one exemplary embodiment of this disclosure, after generating the three-dimensional human body model, the method further includes: obtaining a second hybrid feature corresponding to the image to be processed; and performing orientation estimation on the three-dimensional human body model based on the global depth feature and the second hybrid feature to determine the orientation information of the three-dimensional human body model.

[0012] In an exemplary embodiment of this disclosure, obtaining the second hybrid feature corresponding to the image to be processed includes: extracting features from each instance in the instance segmentation map to obtain instance features; obtaining a local depth map corresponding to the instance, and obtaining local depth features based on the local depth map; and fusing the local depth features and the instance features to obtain the second hybrid feature.

[0013] In an exemplary embodiment of this disclosure, the step of estimating the orientation of the three-dimensional human body model based on global depth features and the second hybrid features to determine the orientation information of the three-dimensional human body model includes: performing a convolution operation on the global depth features and the second hybrid features, and performing a fully connected operation on the convolution result to obtain the orientation information of the target object.

[0014] According to one aspect of this disclosure, a three-dimensional model reconstruction apparatus is provided, comprising: an instance segmentation module for acquiring a depth map of an image to be processed and performing instance segmentation on the image to be processed to obtain an instance segmentation map; a voxel reconstruction module for performing voxel reconstruction on a target object in the image to be processed based on the instance segmentation map to obtain a voxel reconstruction model; an implicit reconstruction module for fitting and calculating voxel features corresponding to the voxel reconstruction model, image features corresponding to the image to be processed, and global depth features corresponding to the depth map to obtain occupancy values ​​of three-dimensional points for implicit reconstruction; and a three-dimensional reconstruction module for performing three-dimensional reconstruction on the target object in the image to be processed based on the occupancy values ​​of the three-dimensional points to obtain a three-dimensional human model of the target object.

[0015] According to one aspect of this disclosure, an electronic device is provided, comprising: a processor; and

[0016] A memory for storing executable instructions of the processor; wherein the processor is configured to execute the three-dimensional model reconstruction method described above by executing the executable instructions.

[0017] The 3D model reconstruction method, 3D model reconstruction device, and electronic device provided in this disclosure, on the one hand, perform voxel 3D reconstruction of the target object in the image to be processed based on the instance segmentation map to reduce the impact of occlusion and pose on the reconstruction. Furthermore, the voxel features output by the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map can be used as input to fit and calculate the occupancy value of 3D points. Based on the 3D points corresponding to the occupancy value, a 3D human model of the target object is constructed. This can reconstruct the model of the occluded, invisible parts, and refine the surface texture through implicit reconstruction representation, improving the accuracy of reconstructing the 3D human model in occluded scenes and achieving refined reconstruction of the 3D human model. On the other hand, it avoids the limitation of related technologies that cannot reconstruct all representations, improving comprehensiveness and increasing the application scope and realism.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0020] Figure 1The flowchart illustrates a three-dimensional model reconstruction method in the related art of this disclosure.

[0021] Figure 2 The flowchart illustrates a three-dimensional model reconstruction method according to an embodiment of the present disclosure.

[0022] Figure 3 The diagram illustrates a voxel reconstruction according to an embodiment of the present disclosure.

[0023] Figure 4 The flowchart illustrating the specific process of obtaining the occupancy value of a three-dimensional point according to an embodiment of the present disclosure is shown.

[0024] Figure 5 The illustration shows a schematic diagram of the process for reconstructing a three-dimensional human body model according to an embodiment of the present disclosure.

[0025] Figure 6 The schematic diagram illustrates a process for obtaining a second hybrid feature according to an embodiment of the present disclosure.

[0026] Figure 7 This schematic diagram illustrates a process for predicting the orientation information of a three-dimensional human body model according to an embodiment of the present disclosure.

[0027] Figure 8 The schematic diagram illustrates the process of three-dimensional model reconstruction according to an embodiment of the present disclosure.

[0028] Figure 9 A schematic block diagram of a three-dimensional model reconstruction apparatus according to an embodiment of the present disclosure is shown.

[0029] Figure 10 A schematic block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0030] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more of these specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0031] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0032] In related technologies, existing single-view multi-person reconstruction methods are based on a parametric reconstruction approach. They predict the parameters of the human body template SMPL (Skinned Multi-Person Linear) and deform it on a basic human body model to achieve 3D reconstruction of the target human body. (Reference) Figure 1 As shown, the specific steps may include: Step S101, acquiring the image to be processed; Step S102, extracting features through an object detection network; Step S103, inputting the features into SMPL parameter regression; Step S104, obtaining a 3D human body model lacking texture details; Step S105, performing geometric constraint loss; Step S106, comparing with a threshold and returning to SMPL parameter regression to continue execution; Step S106, reconstruction complete. This method has the following problems: the human body template-based method only models the human body itself and cannot effectively reconstruct human-related representations such as clothing and hair that are not present in the template; in multi-person scenes, people are prone to mutual occlusion, thus failing to reconstruct the relative spatial relationships between people.

[0033] In this embodiment of the disclosure, to solve the above-mentioned technical problems, a three-dimensional model reconstruction method is provided. (See reference...) Figure 2 As shown, the 3D model reconstruction method mainly includes the following steps:

[0034] In step S210, the depth map of the image to be processed is obtained, and instance segmentation is performed on the image to be processed to obtain an instance segmentation map;

[0035] In step S220, voxel reconstruction is performed on the target object in the image to be processed based on the instance segmentation map to obtain a voxel reconstruction model;

[0036] In step S230, the voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map are fitted and calculated to obtain the occupancy value of the three-dimensional points for implicit reconstruction.

[0037] In step S140, the target object in the image to be processed is reconstructed in three dimensions based on the occupancy value of the three-dimensional points to obtain a three-dimensional human body model of the target object.

[0038] Among these methods, voxel 3D reconstruction still demonstrates good reconstruction performance even when pose, clothing, and hair exhibit significant deformation and occlusion. Therefore, voxel reconstruction is first performed on the input target object using a voxel estimation network to obtain a voxel reconstruction model. Further, due to the low output resolution of voxel reconstruction, it is unable to reconstruct surface texture details. Therefore, secondary detailing is required on the voxel reconstruction model to supplement more refined texture details. Specifically, voxel features are extracted from the voxel 3D model, image features are extracted from the image to be processed, and global depth features are extracted from the depth map corresponding to the image to be processed. A first mixed feature is generated based on the voxel features, image features, and global depth features. Then, the occupancy value of 3D points is predicted based on the first mixed feature using an implicit function. The occupancy value of a 3D point determines whether it is within the target mesh, and a 3D human body model is constructed based on all 3D points within the target mesh.

[0039] Next, refer to Figure 2 The specific steps of the three-dimensional model reconstruction method in the embodiments of this disclosure are explained below.

[0040] In step S210, the depth map of the image to be processed is obtained, and instance segmentation is performed on the image to be processed to obtain an instance segmentation map.

[0041] This disclosure can be applied to augmented reality or virtual reality scenarios, specifically for 3D modeling in such situations, and can also be applied to other application scenarios. The image to be processed can contain multiple objects, such as people, animals, vehicles, or other types of objects; here, a human body is used as an example. The image to be processed can contain multiple human bodies, and due to movement or external environmental conditions, these bodies may undergo significant deformations in various dimensions such as posture, clothing, and hair, resulting in occlusion between the objects. The image to be processed can be an image from a preset scene, such as an occlusion scene. An occlusion scene can be understood as multiple objects in the image to be processed having partially overlapping areas.

[0042] A depth map of the image to be processed can be obtained. A depth map is an image where the distance (depth) from the image acquisition device to each point in the scene is used as pixel values. Each pixel value in the depth map represents the distance from the object to the camera plane and can be used to reflect the geometry of the object's visible surface. The depth map can be obtained by capturing images with a depth camera or by processing each pixel using a depth estimation encoder. In this embodiment, the depth map of the entire image to be processed can be obtained through a depth estimation encoder.

[0043] In scenarios with multiple occlusions, instance segmentation can be performed on the image to be processed to obtain an instance segmentation map corresponding to the image. The target object can be every object contained in the image to be processed, and each object can be segmented into an instance. Instance segmentation is performed on each object in the image, and multiple objects may belong to the same category. Based on this, instance segmentation refers to further refining object detection, separating the foreground and background of objects, and achieving pixel-level object separation. Instance segmentation is used to segment objects of different instances within the same category; it can also be considered that each object will be segmented into an instance. In some embodiments, instance segmentation can represent each object in the image to be processed with a different color and separate each object from the background. Exemplarily, instance segmentation can be performed using any instance segmentation network, such as the Mask-RCNN instance segmentation network or any other type of instance segmentation network, without specific limitations here.

[0044] In step S220, voxel reconstruction is performed on the target object in the image to be processed based on the instance segmentation map to obtain a voxel reconstruction model.

[0045] In this embodiment of the disclosure, voxel 3D reconstruction still exhibits good reconstruction performance even when the pose, clothing, hair have significant deformations, or there is occlusion. Therefore, the voxel 3D model of the input target object is first predicted based on a voxel 3D estimation network.

[0046] A voxel is a concept in three-dimensional space; it is the smallest unit of digital data segmented in three-dimensional space. A voxel itself does not contain positional information; only the relative distances between voxels need to be determined. The voxel method represents the geometry of an object by describing its solid regions in space.

[0047] The three-dimensional space corresponding to the image to be processed can be divided into a series of voxels, for example, 128 voxels. 128 The 128 three-dimensional space is divided into multiple 1 1 A voxel of 1. When reconstructing the target object in the image to be processed based on the instance segmentation map, a voxel estimation network can be used to determine whether the voxels of the target object are on the 3D human body model. Furthermore, a voxel reconstruction model can be constructed based on the voxel's presence state. The presence state can be located on the 3D human body model or not located on the 3D human body model. Based on this, if the presence state is located on the 3D human body model, a voxel reconstruction model can be obtained based on the voxels located on the 3D human body model. This process is repeated, and the presence state of each voxel can be determined using the voxel estimation network, and then a voxel reconstruction model can be constructed based on the presence state and all voxels located on the 3D human body model.

[0048] In this embodiment, a voxel reconstruction model of the instance corresponding to the target object can be output based on a voxel estimation network. The voxel reconstruction model can be an initial 3D model of the target object without details. To avoid technical problems in related technologies, the voxel estimation network in this embodiment can be trained based on 3D reconstruction loss and contour occlusion loss. In some embodiments, a loss function is jointly determined based on the 3D reconstruction loss and contour occlusion loss, and the network parameters of the voxel estimation network are adjusted with the minimum loss function as the training objective to train the voxel estimation network. The loss function can be obtained by weighted summation of the occupancy value of 3D points in the voxel grid, the predicted occupancy value, the real image contour, and the product of the visibility index and the rendered image contour. The specific loss function can be referred to in formula (1):

[0049] Formula (1)

[0050] in, This represents the occupancy value of a 3D point P in the voxel mesh. This represents the predicted occupancy value of a 3D point P in the voxel grid. It is the rendered image outline. 'm' represents the actual image outline, and 'm' is the visibility metric. The visibility metric is used to represent the visibility of the rendered image outline.

[0051] In this embodiment, the actual information and predicted information of the sample voxels can be used as inputs. The network parameters of the voxel estimation network are updated with the goal of minimizing the loss function, resulting in a trained network used for voxel reconstruction. The actual information represents the true occupancy value of a voxel on the 3D human model, while the predicted information represents the predicted occupancy value of a voxel on the 3D human model. By introducing contour occlusion loss into the original 3D reconstruction loss to calculate the loss function, and then training the voxel estimation network based on this loss function, contour occlusion can be introduced into the original 3D model. Voxel reconstruction can be performed from multiple dimensions, including the basic structure and contour occlusion conditions, thus improving the reconstruction effect in occluded scenes.

[0052] For example, refer to Figure 3 As shown, voxel reconstruction can be performed on the three objects contained in the image to be processed in the occluded scene to obtain a voxel reconstruction model corresponding to each object. The three objects in the image to be processed occlude each other. Object 310 corresponds to voxel reconstruction model 311, object 320 corresponds to voxel reconstruction model 321, and object 330 corresponds to voxel reconstruction model 331. The method of constructing the voxel reconstruction model for each object is the same and will not be described further here.

[0053] In this embodiment of the disclosure, by introducing contour occlusion loss on the basis of the original 3D reconstruction loss to calculate the loss function, and then training the voxel estimation network according to the loss function, the accuracy of the voxel estimation network can be improved, thereby improving the accuracy of the voxel reconstruction model of the target object and improving the reconstruction effect in occluded scenes.

[0054] In step S230, the voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map are fitted and calculated to obtain the occupancy value of the three-dimensional points for implicit reconstruction.

[0055] In this embodiment of the disclosure, after obtaining the voxel reconstruction model, since the output resolution of the voxel reconstruction model is low and it is difficult to reconstruct surface texture details, it is necessary to perform detail reconstruction on the surface model represented by the voxel reconstruction model to supplement more refined texture and other details.

[0056] In this process, the voxel features corresponding to the voxel reconstruction model, the image features of the target object in the image to be processed, and the global depth features corresponding to the depth map can be combined for fitting calculation to obtain the occupancy value of each three-dimensional point, thereby performing implicit reconstruction based on voxel reconstruction. Figure 4 The flowchart illustrating the process of obtaining the occupancy value of a 3D point is shown in the figure. (See reference) Figure 4 As shown, the main steps include:

[0057] In step S410, the voxel features corresponding to the voxel reconstruction model, the image features of the image to be processed, and the global depth features corresponding to the depth map are fused to obtain the first hybrid feature;

[0058] In step S420, the first hybrid feature is predicted based on the implicit function to obtain the occupancy value of the three-dimensional point, so as to determine whether the three-dimensional point is within the target grid.

[0059] In some embodiments, voxel features can be obtained from the voxel reconstruction model, image features can be obtained from the image to be processed, and global depth features can be obtained from the depth map. Voxel features can be voxel-related features extracted from the voxel reconstruction model image, image features refer to the basic features of the image itself, and global depth features refer to deeper and more abstract features.

[0060] Furthermore, voxel features, image features, and global depth features can be fused to obtain a first hybrid feature. For example, voxel features, image features, and global depth features can be fused through convolution operations to obtain a first hybrid feature containing features of different dimensions.

[0061] After obtaining the first mixed feature, it can be input into an implicit function. The implicit function then fits and predicts the occupancy value of the 3D point. The implicit function can be any suitable function used to calculate the occupancy value, which indicates whether the 3D point is within the target grid. Fitting and predicting using the implicit function can be understood as fitting the surface of a 3D human body with any suitable implicit function. To improve accuracy, during the process of fitting and calculating the occupancy value of the 3D point using the implicit function, the predicted and actual occupancy values ​​can be supervised using a loss function based on the difference between the predicted and actual occupancy values ​​of the sample 3D points. This can also be understood as updating the parameters of the implicit function based on the loss function to train the implicit function, obtaining more accurate occupancy values ​​and improving accuracy. For example, during the training of the implicit function, the predicted occupancy value can be supervised based on the actual occupancy value of the sample 3D points to achieve supervised implicit function training. Specifically, the predicted occupancy value can be supervised based on the difference between the actual and predicted occupancy values ​​of the sample 3D points to adjust the parameters of the implicit function. For example, if for a sample 3D point A, its actual occupancy value is B and its predicted occupancy value is B', then the predicted occupancy value can be supervised based on the loss function formed by the difference between the actual occupancy value and the predicted occupancy value. The loss function can be shown in formula (2):

[0062] Formula (2)

[0063] In step S240, the target object in the image to be processed is reconstructed in three dimensions based on the occupancy value of the three-dimensional points to obtain a three-dimensional human body model of the target object.

[0064] In this embodiment of the disclosure, after obtaining the occupancy value of a 3D point, it can be determined whether the 3D point is within a target mesh based on the occupancy value. The target mesh can be the voxel mesh described above. The occupancy value can be used to indicate whether the voxel mesh is at least partially occupied by the 3D point. For example, when the occupancy value meets a preset condition, it can be considered to be within the target mesh; when the occupancy value does not meet the preset condition, it can be considered not to be within the target mesh. Meeting the preset condition can include, but is not limited to, any one of: greater than a threshold, equal to a threshold, or within the threshold range, and is not limited here.

[0065] Furthermore, if the 3D points are determined to be within the target grid based on the occupancy value, then the target object in the image to be processed is reconstructed in 3D based on the 3D points within the target grid represented by the occupancy value. Further, the target object in the image to be processed is reconstructed in 3D based on all 3D points within the target grid (i.e., based on all 3D points that meet preset conditions), thereby obtaining a 3D human body model of the target object. This 3D human body model can be a 3D human body model containing detailed information such as texture information. It should be noted that for all target objects in the image to be processed, a corresponding 3D human body model can be constructed using the methods described in steps S210 to S240, thereby obtaining a 3D human body model for each object, i.e., obtaining a multi-person model.

[0066] In this embodiment, since voxel features, image features, and global depth features can be combined to supplement detailed information, complex poses, clothing, and partial occlusion from a single image can be processed without any manual intervention. The occupancy value of three-dimensional points is obtained by combining the voxel reconstruction model, thereby improving the accuracy of the three-dimensional human body model and achieving accurate three-dimensional human body model reconstruction from a single viewpoint.

[0067] Figure 5 The diagram illustrates a flowchart for three-dimensional human body reconstruction. (See reference) Figure 5 As shown, the main steps include:

[0068] In step S510, the image to be processed is acquired. Specifically, it can be an image of an occluded scene, such as the current frame image captured under occlusion conditions.

[0069] In step S520, the depth map and instance segmentation map corresponding to the image to be processed are obtained.

[0070] In step S530, voxel reconstruction is performed based on the instance segmentation map to obtain the voxel reconstruction model.

[0071] In step S540, global depth features are obtained from the depth map, image features are obtained from the image to be processed, and voxel features are obtained from the voxel reconstruction model.

[0072] In step S550, the voxel features corresponding to the voxel reconstruction model, the image features, and the global depth features corresponding to the depth map are fused to obtain the first hybrid feature.

[0073] In step S560, the first hybrid features are reconstructed in three dimensions to obtain a three-dimensional human body model.

[0074] Figure 5 The proposed technical solution, compared to the SMPL template-based method for multi-person 3D reconstruction, first uses voxel representation to process the deformation of pose, clothing, and hair to obtain a voxel reconstruction model. Then, implicit representation is used to refine the surface texture based on the voxel reconstruction model to achieve higher quality textured surface reconstruction. This method can reconstruct occluded, invisible parts and achieve refined surface texture reconstruction, improving reliability, integrity, and comprehensiveness. It not only overcomes the impact of multi-person occlusion scenes on 3D human reconstruction but also preserves the texture details of clothing, hair, and other elements to the maximum extent, making it more widely applicable and practical.

[0075] In this embodiment of the disclosure, after obtaining the 3D human body model of each target image in the image to be processed, in order to ensure the consistency of the spatial position and orientation of the reconstructed person under different viewpoints, it is necessary to predict the spatial position and orientation of each 3D human body in multiple degrees of freedom. Therefore, the orientation information of the 3D human body model can also be determined to predict the spatial position and orientation of the 3D human body model corresponding to each target object. This allows for the implicit reconstruction of the spatially coherent spatial positions and orientations of the target objects and their occluded scenes, thereby reconstructing the relative spatial relationships between multiple people. Multiple degrees of freedom refer to multiple dimensions, such as any one of 3DOF, 6DOF, or 9DOF, specifically determined according to actual needs. Here, 6DOF is used as an example for explanation. In some embodiments, the second mixed feature of the target object in the image to be processed can be obtained; based on the global depth feature and the second mixed feature, the orientation estimation prediction of the target object is performed to determine the orientation information of the 3D human body model.

[0076] Figure 6 A flowchart illustrating the determination of the second mixing characteristic is shown schematically, with reference to Figure 6 As shown, the main steps include:

[0077] In step S610, feature extraction is performed on each instance in the instance segmentation map to obtain instance features;

[0078] In step S620, a depth map corresponding to the instance is obtained, and local depth features are obtained based on the depth map;

[0079] In step S630, the local depth features and the instance features are fused to obtain a second hybrid feature.

[0080] In this embodiment of the disclosure, features can first be extracted from each instance of the target object in the instance segmentation map obtained by instance segmentation to obtain instance features. For example, for each segmented instance, feature extraction can be performed based on a convolutional neural network to obtain instance features for each instance.

[0081] Next, local depth maps corresponding to each instance can be cropped from the depth map, and features can be extracted from these local depth maps using the PointNet network to obtain local depth features. Since each instance is different, the local depth map corresponding to each instance may be different. For example, the depth map can be cropped based on each instance to obtain the local depth map corresponding to instance 1. Figure 1 Local depth corresponding to Example 2 Figure 2 Wait a minute. PointNet uses a hierarchical feature extraction approach, consisting of three parts: a sampling layer, a grouping layer, and a feature extraction layer. First, the sampling layer uses FPS (farthest point sampling) to extract relatively important center points from the dense point cloud. The grouping layer finds the k nearest neighbors within a certain range of the center points extracted in the previous layer to form a patch. The feature extraction layer uses the features obtained by convolution and pooling these k points through a small PointNet network as the features of the center point, and then feeds them into the next layer to obtain the local depth features of each instance.

[0082] Furthermore, local depth features and instance features can be fused to obtain a second hybrid feature. The second hybrid feature can be used for orientation estimation, and its generation method differs from the first hybrid feature. The first hybrid feature is an overall hybrid feature composed of voxel features, image features, and global depth features. The second hybrid feature is a local hybrid feature composed of local depth features and instance features, i.e., a local hybrid feature corresponding to each instance. For example, local depth features and instance features can be input into a hybrid network for convolutional operations to fuse the local depth features and instance features, resulting in the second hybrid feature. The second hybrid feature can be a pixel-level hybrid feature.

[0083] After obtaining the second blended feature, the orientation of the target object can be estimated based on the global depth features obtained from the depth map and the second blended feature to obtain the orientation information of the 3D human body model. Orientation information can include, but is not limited to, spatial location and direction. For example, the global depth features and the second blended feature can be processed by an orientation estimation network. The orientation estimation network can include convolutional networks and fully connected networks. Based on this, a convolutional network can be used to perform convolution operations on the global depth features and the first blended feature to obtain the convolution result, and a fully connected network can be used to perform a fully connected operation on the convolution result, thereby combining the parameters of each dimension to obtain the orientation information of the 3D human body model. The convolution result can be a parameter of one dimension (a parameter of one degree of freedom), used to represent the orientation information in one dimension. Furthermore, a fully connected network can be used to perform a fully connected operation on the parameters of all dimensions to combine them, obtaining multi-dimensional orientation information. Multi-dimensional orientation information can be, for example, 6 degrees of freedom (6DOF) spatial location and direction.

[0084] In this embodiment of the disclosure, the orientation estimation of each three-dimensional human body model is performed by fusing the global depth features obtained from the depth map, the instance features of each instance, and the corresponding local depth features. This can improve the accuracy and comprehensiveness of the orientation information of the three-dimensional human body model.

[0085] Figure 7 This schematically illustrates a flowchart for obtaining the orientation information of a 3D human body model. (Refer to...) Figure 7 As shown, the main steps include:

[0086] In step S710, the image to be processed is acquired.

[0087] In step S720, the depth map and instance segmentation map corresponding to the image to be processed are obtained.

[0088] In step S730, instance features are extracted based on the instance segmentation map, and local depth features are extracted based on the depth map. For example, feature extraction can be performed on each instance in the instance segmentation map to obtain instance features; the depth map is then segmented into local depth maps according to each instance in the depth map, and local depth features are extracted from the local depth maps.

[0089] In step S740, the local depth features obtained from the depth map and the instance features obtained from the instance segmentation map are fused to obtain the second hybrid feature.

[0090] In step S750, the global depth features and the second hybrid features obtained from the depth map are used to predict the orientation information of the 3D human body model. Here, the global depth features can be depth features obtained from the entire depth map, the second hybrid features include local depth features and instance features of the target object, and the orientation information can be the spatial position and orientation of the 3D human body model corresponding to the target object in the image to be processed.

[0091] In this embodiment, local depth features and instance features are integrated, and global depth features are used to improve the orientation estimation process. This enables accurate prediction of the spatial position and orientation of a 3D human body model in occluded scenarios, thus improving the accuracy of orientation estimation. In occluded scenarios, the consistency of the reconstructed 3D human body model's spatial position and orientation under different viewing angles is ensured.

[0092] Figure 8 The diagram illustrates the specific flowchart of 3D model reconstruction, which mainly includes two stages: The first stage involves instance segmentation and depth processing to determine the instance segmentation map and depth map of the input image. In the first stage, a multi-task approach is used to predict the instance segmentation results and the depth information of the target person, extracting the relative spatial position information between target objects in occluded scenes. The second stage is mainly used to realize the 3D human body model reconstruction and orientation estimation. The instance segmentation map and depth map are used in the multi-task learning of the second stage. The first learning task is to achieve refined 3D human body reconstruction under complex poses, clothing, and partial occlusion. The second learning task is to predict the 6DOF spatial position and orientation of the input instance using local and global depth information.

[0093] refer to Figure 8 As shown, the first stage mainly includes steps S802 and S803, and the second stage mainly includes steps S804 and S805. Specifically, the entire process may include the following steps:

[0094] Step S801: Obtain the image to be processed.

[0095] Step S802: Input the image to be processed into the instance segmentation encoder to obtain the instance segmentation map.

[0096] Step S803: Input the image to be processed into the depth estimation encoder to obtain a depth map. The depth estimation encoder can share weights with the instance segmentation encoder. Shared weights mean that the parameters of the instance segmentation encoder and the depth estimation encoder remain constant when traversing the entire image to be processed; that is, all elements of the entire image share the same weights, thereby reducing computational complexity and improving processing efficiency.

[0097] Step S804 involves implicit 3D human body reconstruction using instance segmentation maps and depth maps to obtain a 3D human body model for each target object. First, voxel 3D reconstruction is performed on the target object to reduce the impact of occlusion and pose on the reconstruction. Then, the voxel reconstruction results, image features, and global depth features are used as input to predict the occupancy value of 3D points in a hybrid feature representation, achieving a more accurate secondary implicit reconstruction. The 3D human body model reconstruction process is described in [reference needed]. Figure 5 The steps are shown in the diagram, and will not be repeated here.

[0098] Step S805: Orientation estimation of the 3D human body model is performed using the depth map and instance segmentation map to obtain the orientation information of the 3D human body model. For the detailed process of obtaining the orientation information of the 3D human body model, please refer to [link to relevant documentation]. Figure 7 The steps are shown in the diagram, and will not be repeated here.

[0099] Step S806: Repeatedly execute the three-dimensional human body reconstruction and the orientation estimation process of the three-dimensional human body model to achieve multi-person reconstruction.

[0100] In this embodiment, based on the aforementioned 3D reconstruction flowchart, the reconstruction process of the 3D human body model for each target object under occlusion can be achieved through an instance segmentation module, a depth estimation module, a 3D human body reconstruction module, and a human body spatial position and orientation estimation module. Furthermore, it can reconstruct character-related representations such as clothing and hair that are not present in the template. Through a multi-task learning method and a phased optimization strategy, refined 3D human body reconstruction from a single viewpoint is achieved under complex poses, clothing, and partial occlusion. This not only overcomes the impact of multi-person occlusion scenes on 3D human body reconstruction but also maximizes the preservation of texture details such as clothing and hair, improving accuracy and comprehensiveness. In addition, it reconstructs the relative spatial relationships between different objects, accurately predicting the orientation information of the 3D human body model under occlusion scenes, thus broadening its application scenarios and demonstrating good practicality.

[0101] The technical solution in this embodiment first performs voxel reconstruction on the target object to reduce the impact of occlusion and pose on the reconstruction. Then, the voxel reconstruction model, image features, and global depth features are used as input to predict the occupancy value of 3D points in a hybrid feature representation. Based on the occupancy value of the 3D points, a 3D human body model of the target object is constructed based on the 3D points located in the target mesh. This achieves a more accurate secondary implicit reconstruction representation to refine the surface texture, resulting in higher quality textured surface reconstruction. This method can not only reliably reconstruct occluded invisible parts but also achieve refined reconstruction of surface textures. It can overcome the impact of multi-person occlusion scenes on 3D human body reconstruction and preserve the texture details of clothing, hair, etc., to the maximum extent, thus improving the accuracy and comprehensiveness of 3D human body model reconstruction. In addition, the orientation of the 3D human body model can be estimated based on instance segmentation maps and depth maps. The estimated 6DOF spatial position and orientation can be used to reconstruct the relative spatial position relationships between multiple objects. It can also perform implicit reconstruction of the 6DOF spatial position and orientation of the target person and their occluded scenes. In occluded scenes, it ensures the consistency of the reconstructed person's spatial position and orientation from different perspectives, improves the accuracy of the reconstructed model, enhances the matching between the 3D human body model and the real scene, and increases the realism of the 3D human body model in occluded scenes.

[0102] This disclosure also provides a three-dimensional model reconstruction apparatus. (Reference) Figure 9 As shown, the 3D model reconstruction method 900 mainly includes the following modules:

[0103] The instance segmentation module 901 is used to obtain the depth map of the image to be processed and to perform instance segmentation on the image to be processed to obtain an instance segmentation map;

[0104] Voxel reconstruction module 902 is used to perform voxel reconstruction on the target object in the image to be processed based on the instance segmentation map, and obtain a voxel reconstruction model.

[0105] The implicit reconstruction module 903 is used to fit and calculate the voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map to obtain the occupancy value of the three-dimensional points for implicit reconstruction.

[0106] The 3D reconstruction module 904 is used to perform 3D reconstruction of the target object in the image to be processed based on the occupancy value of the 3D points, and obtain a 3D human body model of the target object.

[0107] In one exemplary embodiment of this disclosure, the step of performing voxel reconstruction on the target object in the image to be processed based on the instance segmentation map to obtain a voxel reconstruction model includes: performing voxel reconstruction on the instance corresponding to the target object in the instance segmentation map through a voxel estimation network to obtain the voxel reconstruction model; wherein the voxel estimation network is trained based on 3D reconstruction loss and contour occlusion loss.

[0108] In an exemplary embodiment of this disclosure, the step of performing voxel reconstruction on the instance corresponding to the target object in the instance segmentation map through a voxel estimation network to obtain the voxel reconstruction model includes: determining whether each voxel of the instance corresponding to the target object is located in the three-dimensional object model through the voxel estimation network to determine the existence state; if the existence state is that the voxel is located in the three-dimensional object model, then constructing the voxel reconstruction model based on the voxel.

[0109] In one exemplary embodiment of this disclosure, the step of fitting and calculating the voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map to obtain the occupancy value of the three-dimensional point for implicit reconstruction includes: fusing the voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map to obtain a first hybrid feature; predicting the first hybrid feature based on the implicit function to obtain the occupancy value of the three-dimensional point, so as to determine whether the three-dimensional point is within the target grid.

[0110] In one exemplary embodiment of this disclosure, the method further includes: monitoring the predicted occupancy value and the actual occupancy value based on the difference between the predicted occupancy value and the actual occupancy value.

[0111] In one exemplary embodiment of this disclosure, after generating the three-dimensional human body model, the method further includes: obtaining a second hybrid feature corresponding to the image to be processed; and performing orientation estimation on the three-dimensional human body model based on the global depth feature and the second hybrid feature to determine the orientation information of the three-dimensional human body model.

[0112] In an exemplary embodiment of this disclosure, obtaining the second hybrid feature corresponding to the image to be processed includes: extracting features from each instance in the instance segmentation map to obtain instance features; obtaining a local depth map corresponding to the instance, and obtaining local depth features based on the local depth map; and fusing the local depth features and the instance features to obtain the second hybrid feature.

[0113] In an exemplary embodiment of this disclosure, the step of estimating the orientation of the three-dimensional human body model based on global depth features and the second hybrid features to determine the orientation information of the three-dimensional human body model includes: performing a convolution operation on the global depth features and the second hybrid features, and performing a fully connected operation on the convolution result to obtain the orientation information of the target object.

[0114] It should be noted that the specific details of each module in the above-mentioned 3D model reconstruction device have been described in detail in the corresponding 3D model reconstruction method, so they will not be repeated here.

[0115] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0116] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0117] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0118] Those skilled in the art will understand that various aspects of this disclosure can be implemented as systems, methods, or program products. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: entirely in hardware, entirely in software (including firmware, microcode, etc.), or in a combination of hardware and software, collectively referred to herein as “circuit,” “module,” or “system.”

[0119] The following reference Figure 10 To describe an electronic device 1000 according to such an embodiment of the present disclosure. Figure 10 The electronic device 1000 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0120] like Figure 10As shown, the electronic device 1000 is manifested in the form of a general-purpose computing device. The components of the electronic device 1000 may include, but are not limited to: at least one processing unit 1010, at least one storage unit 1020, a bus 1030 connecting different system components (including storage unit 1020 and processing unit 1010), and a display unit 1040.

[0121] The storage unit stores program code that can be executed by the processing unit 1010, causing the processing unit 1010 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 1010 can perform actions such as... Figure 2 The steps are shown in the figure.

[0122] Storage unit 1020 may include readable media in the form of volatile storage units, such as random access memory (RAM) 10201 and / or cache memory 10202, and may further include read-only memory (ROM) 10203.

[0123] Storage unit 1020 may also include a program / utility 10204 having a set (at least one) program module 10205, such program module 10205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0124] Bus 1030 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the multiple bus structures.

[0125] Electronic device 1000 can also communicate with one or more external devices 1100 (e.g., keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1000, and / or any device that enables electronic device 1000 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 1050. Furthermore, electronic device 1000 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1060. As shown, network adapter 1060 communicates with other modules of electronic device 1000 via bus 1030. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1000, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0126] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or electronic device, etc.) to execute the methods according to the embodiments of this disclosure.

[0127] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of this disclosure described in the "Exemplary Methods" section above.

[0128] The program product for implementing the above-described method according to embodiments of the present disclosure may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0129] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0130] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0131] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0132] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0133] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0134] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention described herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not invented by this disclosure. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A method for reconstructing a three-dimensional model, characterized in that, include: Obtain the depth map of the image to be processed, and perform instance segmentation on the image to be processed to obtain the instance segmentation map; The voxel estimation network determines whether each voxel of the instance corresponding to the target object in the instance segmentation map is located in the 3D object model to determine the existence state, wherein the existence state is that the voxel is located in the 3D object model, and the voxel reconstruction model is obtained based on the voxel; the voxel estimation network is trained based on the 3D reconstruction loss and the contour occlusion loss. The voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map are fitted and calculated to obtain the occupancy value of the three-dimensional points for implicit reconstruction. Based on the occupancy value of the three-dimensional points, the target object in the image to be processed is reconstructed in three dimensions to obtain a three-dimensional human body model of the target object. The step of fitting and calculating the voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map to obtain the occupancy value of the 3D points for implicit reconstruction includes: The voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map are fused to obtain the first hybrid feature; Based on the implicit function, the first hybrid feature is predicted to obtain the occupancy value of the three-dimensional point, so as to determine whether the three-dimensional point is within the target grid.

2. The three-dimensional model reconstruction method according to claim 1, characterized in that, The method further includes: The predicted occupancy value and the actual occupancy value are monitored based on the difference between the predicted occupancy value and the actual occupancy value.

3. The three-dimensional model reconstruction method according to claim 1, characterized in that, After generating the three-dimensional human body model, the method further includes: Obtain the second blending feature corresponding to the image to be processed; Based on the global depth features and the second hybrid features, the orientation of the three-dimensional human body model is estimated to determine the orientation information of the three-dimensional human body model.

4. The three-dimensional model reconstruction method according to claim 3, characterized in that, The step of obtaining the second mixed feature corresponding to the image to be processed includes: Feature extraction is performed on each instance in the instance segmentation map to obtain instance features; Obtain the local depth map corresponding to the instance, and obtain local depth features based on the local depth map; The local depth features and the instance features are fused to obtain a second hybrid feature.

5. The three-dimensional model reconstruction method according to claim 3, characterized in that, The step of estimating the orientation of the 3D human body model based on the global depth features and the second hybrid features, and determining the orientation information of the 3D human body model, includes: The global depth features and the second hybrid features are convolved, and the convolution results are fully connected to obtain the orientation information of the target object.

6. A three-dimensional model reconstruction device, characterized in that, include: The instance segmentation module is used to obtain the depth map of the image to be processed and to perform instance segmentation on the image to be processed to obtain the instance segmentation map; The voxel reconstruction module is used to determine whether each voxel of the instance corresponding to the target object in the instance segmentation map is located in the 3D object model to determine the existence state, wherein the existence state is that the voxel is located in the 3D object model, and to obtain the voxel reconstruction model based on the voxel; the voxel estimation network is trained based on the 3D reconstruction loss and the contour occlusion loss. The implicit reconstruction module is used to fit and calculate the voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map to obtain the occupancy value of the three-dimensional points for implicit reconstruction. The 3D reconstruction module is used to perform 3D reconstruction of the target object in the image to be processed based on the occupancy value of the 3D points, and obtain a 3D human body model of the target object; The step of fitting and calculating the voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map to obtain the occupancy value of the 3D points for implicit reconstruction includes: The voxel features corresponding to the voxel reconstruction model, the image features corresponding to the image to be processed, and the global depth features corresponding to the depth map are fused to obtain the first hybrid feature; Based on the implicit function, the first hybrid feature is predicted to obtain the occupancy value of the three-dimensional point, so as to determine whether the three-dimensional point is within the target grid.

7. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the three-dimensional model reconstruction method of any one of claims 1 to 5 by executing the executable instructions.

Images