Three-dimensional reconstruction method and device, and computer storable medium

Abstract

The present disclosure relates to a three-dimensional reconstruction method and device, a computer storable medium, and the field of computer technology. The three-dimensional reconstruction method comprises: obtaining an image to be processed, wherein the image to be processed comprises a plurality of objects to be reconstructed; determining the initial three-dimensional shape and posture parameters of each object to be reconstructed using a first neural network according to the image to be processed; adjusting the initial three-dimensional shape and posture parameters of each object to be reconstructed using a second neural network to obtain the target three-dimensional shape and posture parameters of each object to be reconstructed; determining the three-dimensional model corresponding to each object to be reconstructed according to the target three-dimensional shape and posture parameters of each object to be reconstructed. According to the present disclosure, the accuracy of three-dimensional reconstruction can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a three-dimensional reconstruction method and device, and a computer storable medium. Background Art

[0002] In the related art, the image features of each object to be reconstructed in an image including multiple objects to be reconstructed are first extracted, and then the image features of each object to be reconstructed are regressed to obtain the corresponding three-dimensional shape and posture parameters, and finally the three-dimensional model of each object to be reconstructed is determined based on the three-dimensional shape and posture parameters. Summary of the invention

[0003] In a 3D reconstruction scene that includes multiple objects to be reconstructed in a single image, there are usually complex situations such as occlusion and crowding between the multiple objects to be reconstructed, and related technologies cannot accurately perform 3D reconstruction of such a scene.

[0004] In view of the above technical problems, the present disclosure proposes a solution to improve the accuracy of three-dimensional reconstruction.

[0005] According to a first aspect of the present disclosure, a three-dimensional reconstruction method is provided, comprising: acquiring an image to be processed, the image to be processed comprising a plurality of objects to be reconstructed; determining, based on the image to be processed, an initial three-dimensional shape and posture parameters of each object to be reconstructed using a first neural network; adjusting, using a second neural network, the initial three-dimensional shape and posture parameters of each object to be reconstructed to obtain a target three-dimensional shape and posture parameters of each object to be reconstructed; and determining, based on the target three-dimensional shape and posture parameters of each object to be reconstructed, a three-dimensional model corresponding to each object to be reconstructed.

[0006] In some embodiments, determining the initial three-dimensional shape and posture parameters of each object to be reconstructed includes: extracting image features of each object to be reconstructed using the first neural network based on the image to be processed; determining the initial three-dimensional shape and posture parameters of each object to be reconstructed using the first neural network based on the corresponding image features for each object to be reconstructed; adjusting the initial three-dimensional shape and posture parameters of each object to be reconstructed includes: adjusting the initial three-dimensional shape and posture parameters of each object to be reconstructed using the second neural network based on the image features of each object to be reconstructed extracted by the first neural network and the corresponding initial three-dimensional shape and posture parameters determined by the first neural network.

[0007] In some embodiments, the image features include a feature map, and the 3D reconstruction method further includes: projecting the initial 3D shape and posture parameters of each object to be reconstructed onto the corresponding feature map to obtain a projection area; for each object to be reconstructed, sampling is performed on the corresponding projected feature map based on a preset sampling range to obtain sampling features, and the preset sampling range is the projection area and the surrounding area within a preset pixel range around the projection area; wherein, according to the initial 3D shape and posture parameters and image features of each object to be reconstructed, adjusting the initial 3D shape and posture parameters of each object to be reconstructed includes: for each object to be reconstructed, according to the corresponding sampling features and initial 3D shape and posture parameters, using the second neural network, adjusting the initial 3D shape and posture parameters of each object to be reconstructed to obtain the target 3D shape and posture parameters.

[0008] In some embodiments, the second neural network includes a feature extraction network and a regression network. According to the corresponding sampling features and the initial three-dimensional shape and posture parameters, the second neural network is used to adjust the initial three-dimensional shape and posture parameters of each object to be reconstructed, so as to obtain the target three-dimensional shape and posture parameters. The method comprises: for each object to be reconstructed, according to the corresponding sampling features, using the feature extraction network of the second neural network to determine the target feature vector; for each object to be reconstructed, according to the corresponding initial three-dimensional shape and posture parameters and the target feature vector, using the regression network of the second neural network to determine the target three-dimensional shape and posture parameters.

[0009] In some embodiments, the feature extraction network of the second neural network is a multilayer perceptron or a fully connected neural network.

[0010] In some embodiments, adjusting the initial three-dimensional shape and posture parameters of each object to be reconstructed includes: projecting the multiple objects to be reconstructed into the same reference two-dimensional image based on the initial parameter information of the multiple objects to be reconstructed; determining the overlap rate of the multiple objects to be reconstructed in the reference two-dimensional image; and, when the overlap rate is greater than an overlap rate threshold, using a second neural network to adjust the initial three-dimensional shape and posture parameters of each object to be reconstructed to obtain the target three-dimensional shape and posture parameters.

[0011] In some embodiments, the overlap ratio is a ratio of the overlapped areas of the multiple objects to be reconstructed in the reference two-dimensional image to their total areas in the reference two-dimensional image.

[0012] In some embodiments, the first neural network includes a feature extraction network and a regression network, and determining the initial three-dimensional shape and posture parameters of each object to be reconstructed includes: extracting image features of each object to be reconstructed using the feature extraction network of the first neural network according to the image to be processed; and determining the initial three-dimensional shape and posture parameters of each object to be reconstructed according to the corresponding image features using the regression network of the first neural network.

[0013] In some embodiments, the image features include a feature map, and using the regression network of the first neural network to determine the initial three-dimensional shape and posture parameters of each object to be reconstructed includes: determining the feature vector of the corresponding feature map for each object to be reconstructed; and determining the initial three-dimensional shape and posture parameters of each object to be reconstructed using the regression network of the first neural network based on the feature vector corresponding to each object to be reconstructed.

[0014] In some embodiments, the feature extraction network of the first neural network is a regional convolutional neural network or a central network.

[0015] In some embodiments, the first neural network and the second neural network are different neural networks.

[0016] According to a second aspect of the present disclosure, a three-dimensional reconstruction device is provided, comprising: an acquisition module, configured to acquire an image to be processed, wherein the image to be processed includes a plurality of objects to be reconstructed; a first determination module, configured to determine, based on the image to be processed and using a first neural network, an initial three-dimensional shape and posture parameters of each object to be reconstructed; an adjustment module, configured to adjust the initial three-dimensional shape and posture parameters of each object to be reconstructed and obtain a target three-dimensional shape and posture parameters of each object to be reconstructed by using a second neural network; and a second determination module, configured to determine, based on the target three-dimensional shape and posture parameters of each object to be reconstructed, a three-dimensional model corresponding to each object to be reconstructed.

[0017] According to a third aspect of the present disclosure, a three-dimensional reconstruction device is provided, comprising: a memory; and a processor coupled to the memory, wherein the processor is configured to execute the three-dimensional reconstruction method described in any of the above embodiments based on instructions stored in the memory.

[0018] According to a fourth aspect of the present disclosure, a computer storable medium is provided, on which computer program instructions are stored, and when the instructions are executed by a processor, the three-dimensional reconstruction method described in any of the above embodiments is implemented.

[0019] In the above embodiments, the accuracy of three-dimensional reconstruction can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0021] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0022] Figure 1 is a flowchart illustrating a three-dimensional reconstruction method according to some embodiments of the present disclosure;

[0023] Figure 2 is a schematic diagram illustrating a three-dimensional reconstruction method according to some embodiments of the present disclosure;

[0024] Figure 3 is a block diagram showing a three-dimensional reconstruction apparatus according to some embodiments of the present disclosure;

[0025] Figure 4 is a block diagram showing a three-dimensional reconstruction apparatus according to some embodiments of the present disclosure;

[0026] Figure 5 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0027] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0028] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0029] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0030] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0031] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0032] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0033] Figure 1 is a flowchart illustrating a three-dimensional reconstruction method according to some embodiments of the present disclosure.

[0034] like Figure 1 As shown, the three-dimensional reconstruction method includes steps S1 to S4.

[0035] In step S1, an image to be processed is obtained. The image to be processed includes a plurality of objects to be reconstructed.

[0036] In step S2, the initial three-dimensional shape and posture parameters of each object to be reconstructed are determined using the first neural network according to the image to be processed. In some embodiments, the three-dimensional shape and posture parameters characterize the three-dimensional shape and posture of the object to be reconstructed. For example, the three-dimensional shape and posture parameters are SMPL (Skinned Multi-Person Linear) model parameters. Taking the object to be reconstructed as a human body as an example, the shape parameters control the height, weight, and thinness of the human body; the posture parameters are generally represented by the rotation angles of the 24 joints of the human body, and the posture parameters control the posture movements of the human body.

[0037] In some embodiments, the image features of each object to be reconstructed are extracted using a first neural network based on the image to be processed. And for each object to be reconstructed, the initial three-dimensional shape and posture parameters of each object to be reconstructed are determined using the first neural network based on the corresponding image features. The image features characterize the object features of each object to be reconstructed. For example, when the object to be reconstructed is a human body, the image features characterize the human body features of the human body. In some embodiments, the image features include feature maps.

[0038] In some embodiments, the first neural network includes a first feature extraction network and a first regression network. First, according to the image to be processed, the first feature extraction network of the first neural network is used to extract the image features of each object to be reconstructed. Then, for each object to be reconstructed, according to the corresponding image features, the first regression network of the first neural network is used to determine the initial three-dimensional shape and posture parameters of each object to be reconstructed.

[0039] Taking the image features including the feature map as an example, determining the initial three-dimensional shape and posture parameters of each object to be reconstructed by using the first regression network of the first neural network includes: determining the feature vector of the corresponding feature map for each object to be reconstructed; and determining the initial three-dimensional shape and posture parameters of each object to be reconstructed by using the first regression network of the first neural network according to the feature vector corresponding to each object to be reconstructed. For example, the feature vectorization module of the first neural network is used to determine the feature vector of the feature map.

[0040] In some embodiments, the first feature extraction network of the first neural network is a regional convolutional neural network (R-CNN) or a center network (CenterNet).

[0041] In step S3, the initial three-dimensional shape and posture parameters of each object to be reconstructed are adjusted using the second neural network to obtain the target three-dimensional shape and posture parameters of each object to be reconstructed. The accuracy of three-dimensional reconstruction can be improved by cascading the first neural network and the second neural network, performing initial processing by the first neural network, and then further optimizing and adjusting the results of the initial processing by the second neural network.

[0042] In some embodiments, the second neural network is different from the first neural network, which may be different in parameters or network structure.

[0043] In some embodiments, the initial three-dimensional shape and posture parameters of each object to be reconstructed are adjusted using the second neural network based on the image features of each object to be reconstructed extracted by the first neural network and the corresponding initial three-dimensional shape and posture parameters determined by the first neural network. By adjusting the initial three-dimensional shape and posture parameters in combination with the intermediate results (i.e., the extracted image features) obtained by using the first neural network, the focus on the features of each object to be reconstructed itself can be increased, thereby further improving the accuracy of three-dimensional reconstruction.

[0044] In some embodiments, taking the image features including feature maps as an example, before adjusting the initial three-dimensional shape and posture parameters, the initial three-dimensional shape and posture parameters of each object to be reconstructed are projected onto the corresponding feature map to obtain a projection area. For each object to be reconstructed, based on a preset sampling range, sampling is performed on the corresponding projected feature map to obtain sampling features. The preset sampling range is the projection area and the surrounding area within the preset pixel range around it. For example, the K-fold pixel range around the area where the projection pixel is located. The value of K is 1.1 to 1.5. Compared with the feature map, these sampling points have a finer-grained characterization capability, can more finely describe the features of the object to be reconstructed, and can more accurately locate the position of the object to be reconstructed in the image.

[0045] In some embodiments, the three-dimensional shape and posture parameters are projected onto a two-dimensional feature map according to camera parameters. The camera parameters include size parameters and translation parameters. The size parameter is the size of the human body in the image, and the translation parameter reflects the translation range of the human body in the image relative to the center of the image.

[0046] Then, for each object to be reconstructed, the initial three-dimensional shape and posture parameters of each object to be reconstructed are adjusted using the second neural network according to the corresponding sampling features and the initial three-dimensional shape and posture parameters to obtain the target three-dimensional shape and posture parameters. In the above embodiment, the diffusion sampling is centered on the projection area of ​​the initial three-dimensional shape and posture parameters in the feature map, and some closely related features can be added on the basis of the initial three-dimensional shape and posture parameters, thereby improving the attention and completeness of the features of the object to be reconstructed itself, thereby improving the accuracy of three-dimensional reconstruction.

[0047] In some embodiments, the second neural network includes a second feature extraction network and a second regression network. For each object to be reconstructed, the target feature vector is determined using the second feature extraction network of the second neural network according to the corresponding sampling features. For each object to be reconstructed, the target three-dimensional shape and posture parameters are determined using the second regression network of the second neural network according to the corresponding initial three-dimensional shape and posture parameters and the target feature vector.

[0048] In some embodiments, the second feature extraction network of the second neural network is a multilayer perceptron (MLP) or a fully connected neural network.

[0049] In some embodiments, based on the initial parameter information of the multiple objects to be reconstructed, the multiple objects to be reconstructed are projected into the same reference two-dimensional image; the overlap rate of the multiple objects to be reconstructed in the reference two-dimensional image is determined; when the overlap rate is greater than the overlap rate threshold, the initial three-dimensional shape and posture parameters of each object to be reconstructed are adjusted using a second neural network to obtain the target three-dimensional shape and posture parameters. For example, the overlap rate threshold is 0.2.

[0050] For example, the overlap ratio is the ratio of the overlap area of ​​the multiple objects to be reconstructed in the reference two-dimensional image to the total area of ​​the multiple objects to be reconstructed in the reference two-dimensional image. For another example, the overlap ratio can also be the ratio of the number of the objects to be reconstructed with overlapping areas in the multiple objects to be reconstructed in the reference two-dimensional image to the total number of the multiple objects to be reconstructed.

[0051] In the above embodiment, by performing three-dimensional reconstruction only on the object to be reconstructed in the image to be processed whose overlap rate is greater than the overlap rate threshold, both the efficiency and accuracy of the three-dimensional reconstruction are taken into account.

[0052] In step S4, a three-dimensional model corresponding to each object to be reconstructed is determined according to the target three-dimensional shape and posture parameters of each object to be reconstructed.

[0053] In some embodiments, the loss functions in training the first neural network and the second neural network are both the square root error (MSE) between the estimated values ​​of the three-dimensional shape and posture parameters and the actual label paper, which belongs to supervised training.

[0054] Figure 2 is a schematic diagram illustrating a three-dimensional reconstruction method according to some embodiments of the present disclosure.

[0055] like Figure 2 As shown, the image to be processed includes two human bodies as two objects to be reconstructed.

[0056] The image to be processed is input into the first feature extraction network of the first neural network to obtain feature maps F1 and F2 of two human bodies.

[0057] For the feature graphs F1 and F2, the corresponding feature vectors V1 and V2 are determined respectively. The feature vectors V1 and V2 are respectively input into the corresponding first regression network to obtain the corresponding initial three-dimensional shape and posture parameters P1 and P2 ( Figure 2 In order to be more intuitive, an initial three-dimensional model corresponding to the parameters is constructed).

[0058] According to the initial 3D shape and posture parameters P1 and P2, the two human bodies are projected into the same reference 2D image, and the overlap rate of the two human bodies in the reference 2D image is calculated.

[0059] For example, the overlap rate in the reference two-dimensional image is greater than an overlap rate threshold. Figure 2 Take a human body as an example to illustrate the adjustment steps. Project the initial 3D shape and posture parameter P1 onto the corresponding feature map F1. Sampling is performed on the projected feature map F1 to obtain the sampling features. For intuitive representation, Figure 2 The sampled features are shown on a human body model, and it can be seen that the sampled features have a wider range than the initial 3D shape and posture parameters. The feature map F1 covers a larger area and contains information about person P2 that is occluded by person P1; and the new features obtained after projecting P1 onto F1 focus more on person P1.

[0060] The sampled features are input into the second feature extraction network of the second neural network to obtain the target feature vector V1'. The target feature vector V1' and the initial three-dimensional shape and posture parameters P1 are input into the second regression network of the second neural network to obtain the target three-dimensional shape and posture parameters from Figure 2 It can be seen that the target 3D shape and posture parameters The corresponding three-dimensional model is closer to the shape and posture of the human body corresponding to the image to be processed.

[0061] Figure 3 is a block diagram showing a 3D reconstruction apparatus according to some embodiments of the present disclosure.

[0062] like Figure 3 As shown, the three-dimensional reconstruction device 3 includes an acquisition module 31 , a first determination module 32 , an adjustment module 33 and a second determination module 34 .

[0063] The acquisition module 31 is configured to acquire the image to be processed, for example, by performing the following steps: Figure 1 As shown in step S1, the image to be processed includes a plurality of objects to be reconstructed.

[0064] The first determination module 32 is configured to determine the initial three-dimensional shape and posture parameters of each object to be reconstructed based on the image to be processed by using the first neural network, for example, performing the following steps: Figure 1 Step S2 is shown.

[0065] The adjustment module 33 is configured to use the second neural network to adjust the initial three-dimensional shape and posture parameters of each object to be reconstructed to obtain the target three-dimensional shape and posture parameters of each object to be reconstructed, for example, by performing the following steps: Figure 1 Step S3 shown.

[0066] The second determination module 34 is configured to determine the three-dimensional model corresponding to each object to be reconstructed according to the target three-dimensional shape and posture parameters of each object to be reconstructed, for example, by performing the following steps: Figure 1 Step S4 is shown.

[0067] Figure 4 is a block diagram showing a 3D reconstruction apparatus according to some embodiments of the present disclosure.

[0068] like Figure 4 As shown, the three-dimensional reconstruction device 4 includes a memory 41; and a processor 42 coupled to the memory 41. The memory 41 is used to store instructions for executing the corresponding embodiments of the three-dimensional reconstruction method. The processor 42 is configured to execute the three-dimensional reconstruction method in any of the embodiments of the present disclosure based on the instructions stored in the memory 41.

[0069] Figure 5 is a block diagram illustrating a computer system for implementing some embodiments of the present disclosure.

[0070] like Figure 5 As shown, the computer system 50 may be embodied in the form of a general purpose computing device. The computer system 50 includes a memory 510, a processor 520, and a bus 500 that connects the various system components.

[0071] The memory 510 may include, for example, a system memory, a non-volatile storage medium, etc. The system memory may store, for example, an operating system, an application program, a boot loader, and other programs. The system memory may include a volatile storage medium, such as a random access memory (RAM) and / or a cache memory. The non-volatile storage medium may store, for example, instructions for executing at least one corresponding embodiment of the three-dimensional reconstruction method. The non-volatile storage medium includes, but is not limited to, a disk memory, an optical memory, a flash memory, etc.

[0072] The processor 520 can be implemented by a general processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistors, etc. Discrete hardware components. Accordingly, each module such as the judgment module and the determination module can be implemented by a central processing unit (CPU) running instructions in a memory that execute corresponding steps, or can be implemented by a dedicated circuit that executes corresponding steps.

[0073] The bus 500 may use any of a variety of bus architectures, including, but not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, and a Peripheral Component Interconnect (PCI) bus.

[0074] The computer system 50 may also include an input / output interface 530, a network interface 540, a storage interface 550, etc. These interfaces 530, 540, 550, the memory 510, and the processor 520 may be connected via a bus 500. The input / output interface 530 may provide a connection interface for input / output devices such as a display, a mouse, and a keyboard. The network interface 540 may provide a connection interface for various networked devices. The storage interface 550 may provide a connection interface for external storage devices such as a floppy disk, a USB flash drive, and an SD card.

[0075] Here, various aspects of the present disclosure are described with reference to flowcharts and / or block diagrams of methods, devices, and computer program products according to embodiments of the present disclosure. It should be understood that each frame of the flowchart and / or block diagram and the combination of frames can be implemented by computer-readable program instructions.

[0076] These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer or other programmable device to produce a machine, so that the processor executes the instructions to produce means for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0077] These computer-readable program instructions may also be stored in a computer-readable memory, which cause the computer to work in a specific manner to produce an article of manufacture, including instructions for implementing the functions specified in one or more blocks in the flowchart and / or block diagram.

[0078] The present disclosure can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects.

[0079] The accuracy of three-dimensional reconstruction can be improved by using the three-dimensional reconstruction method and device and computer storable medium in the above embodiments.

[0080] So far, the three-dimensional reconstruction method and device, and computer storable medium according to the present disclosure have been described in detail. In order to avoid obscuring the concept of the present disclosure, some details known in the art are not described. Based on the above description, those skilled in the art can fully understand how to implement the technical solution disclosed here.

Claims

1. A three-dimensional reconstruction method, comprising: Acquire an image to be processed, wherein the image to be processed includes a plurality of objects to be reconstructed; Determining the initial three-dimensional shape and posture parameters of each object to be reconstructed according to the image to be processed using the first neural network, including: extracting image features of each object to be reconstructed according to the image to be processed using the first neural network, the image features including feature maps, and determining the initial three-dimensional shape and posture parameters of each object to be reconstructed according to the corresponding image features using the first neural network for each object to be reconstructed; Projecting the initial three-dimensional shape and posture parameters of each object to be reconstructed onto the corresponding feature map to obtain a projection area, and for each object to be reconstructed, sampling is performed on the corresponding projected feature map based on a preset sampling range to obtain sampling features, wherein the preset sampling range is the projection area and the surrounding area within a preset pixel range around it; Using the second neural network, adjusting the initial three-dimensional shape and posture parameters of each object to be reconstructed to obtain the target three-dimensional shape and posture parameters of each object to be reconstructed, including: for each object to be reconstructed, according to the corresponding sampling features and the initial three-dimensional shape and posture parameters, using the second neural network, adjusting the initial three-dimensional shape and posture parameters of each object to be reconstructed to obtain the target three-dimensional shape and posture parameters; According to the target three-dimensional shape and posture parameters of each object to be reconstructed, a three-dimensional model corresponding to each object to be reconstructed is determined.

2. The three-dimensional reconstruction method according to claim 1, wherein: Adjusting the initial 3D shape and posture parameters of each object to be reconstructed includes: The initial three-dimensional shape and posture parameters of each object to be reconstructed are adjusted using the second neural network according to the image features of each object to be reconstructed extracted by the first neural network and the corresponding initial three-dimensional shape and posture parameters determined by the first neural network.

3. The three-dimensional reconstruction method according to claim 1, wherein: The second neural network includes a feature extraction network and a regression network. According to the corresponding sampling features and the initial three-dimensional shape and posture parameters, the initial three-dimensional shape and posture parameters of each object to be reconstructed are adjusted by using the second neural network to obtain the target three-dimensional shape and posture parameters including: For each object to be reconstructed, according to the corresponding sampling features, a feature extraction network of the second neural network is used to determine a target feature vector; For each object to be reconstructed, the target three-dimensional shape and posture parameters are determined using the regression network of the second neural network according to the corresponding initial three-dimensional shape and posture parameters and the target feature vector.

4. The three-dimensional reconstruction method according to claim 3, wherein: The feature extraction network of the second neural network is a multilayer perceptron or a fully connected neural network.

5. The three-dimensional reconstruction method according to claim 1, wherein: Adjusting the initial 3D shape and posture parameters of each object to be reconstructed includes: projecting the multiple objects to be reconstructed into a same reference two-dimensional image according to the initial parameter information of the multiple objects to be reconstructed; determining an overlap ratio of the plurality of objects to be reconstructed in the reference two-dimensional image; When the overlap rate is greater than the overlap rate threshold, the second neural network is used to adjust the initial three-dimensional shape and posture parameters of each object to be reconstructed to obtain the target three-dimensional shape and posture parameters.

6. The three-dimensional reconstruction method according to claim 5, wherein: The overlap ratio is a ratio of the overlapped areas of the multiple objects to be reconstructed in the reference two-dimensional image to their total areas in the reference two-dimensional image.

7. The three-dimensional reconstruction method according to claim 1, wherein: The first neural network includes a feature extraction network and a regression network, and determining the initial three-dimensional shape and posture parameters of each object to be reconstructed includes: According to the image to be processed, using the feature extraction network of the first neural network, extracting image features of each object to be reconstructed; For each object to be reconstructed, according to the corresponding image features, the regression network of the first neural network is used to determine the initial three-dimensional shape and posture parameters of each object to be reconstructed.

8. The three-dimensional reconstruction method according to claim 7, wherein: The image features include a feature map, and using the regression network of the first neural network, determining the initial three-dimensional shape and posture parameters of each object to be reconstructed includes: For each object to be reconstructed, determining a feature vector of a corresponding feature map; According to the feature vector corresponding to each object to be reconstructed, the initial three-dimensional shape and posture parameters of each object to be reconstructed are determined using the regression network of the first neural network.

9. The three-dimensional reconstruction method according to claim 7, wherein: The feature extraction network of the first neural network is a regional convolutional neural network or a central network.

10. The three-dimensional reconstruction method according to claim 1, wherein: The first neural network and the second neural network are different neural networks.

11. A three-dimensional reconstruction device, comprising: An acquisition module is configured to acquire an image to be processed, wherein the image to be processed includes a plurality of objects to be reconstructed; The first determination module is configured to determine the initial three-dimensional shape and posture parameters of each object to be reconstructed according to the image to be processed by using a first neural network, including: extracting image features of each object to be reconstructed according to the image to be processed by using the first neural network, the image features including feature maps, and determining the initial three-dimensional shape and posture parameters of each object to be reconstructed according to the corresponding image features by using the first neural network for each object to be reconstructed; The adjustment module is configured to project the initial three-dimensional shape and posture parameters of each object to be reconstructed onto a corresponding feature map to obtain a projection area, and for each object to be reconstructed, based on a preset sampling range, sample on the corresponding projected feature map to obtain sampling features, wherein the preset sampling range is the projection area and a surrounding area within a preset pixel range around the projection area, and use a second neural network to adjust the initial three-dimensional shape and posture parameters of each object to be reconstructed to obtain a target three-dimensional shape and posture parameters of each object to be reconstructed, including: for each object to be reconstructed, according to the corresponding sampling features and the initial three-dimensional shape and posture parameters, use the second neural network to adjust the initial three-dimensional shape and posture parameters of each object to be reconstructed to obtain the target three-dimensional shape and posture parameters; The second determination module is configured to determine a three-dimensional model corresponding to each object to be reconstructed according to a target three-dimensional shape and posture parameters of each object to be reconstructed.

12. A three-dimensional reconstruction device, comprising: Memory; as well as A processor coupled to the memory, wherein the processor is configured to execute the three-dimensional reconstruction method according to any one of claims 1 to 10 based on instructions stored in the memory.

13. A computer storable medium having computer program instructions stored thereon, which, when executed by a processor, implement the three-dimensional reconstruction method according to any one of claims 1 to 10.

Images