Method and apparatus for generating three-dimensional avatar, and device and storage medium

Abstract

On the basis of the embodiments of the present disclosure, provided are a method and apparatus for generating a three-dimensional avatar, and a device and a storage medium. The method comprises: on the basis of an input feature with respect to a target object, generating reference information in a reference space, which reference information corresponds to a reference form, wherein the reference information indicates a reference surface of the target object in the reference form; determining a target form of the target object; and on the basis of the reference information and a transformation relationship between the target form and the reference form, generating three-dimensional avatar information of the target object in the target form. In this way, the generation of high-quality three-dimensional avatars in various desired forms can be advantageously realized.

Description

Method, device, equipment and storage medium for generating three-dimensional image

[0001] This application claims priority to the Chinese invention patent application entitled “Method, device, apparatus and storage medium for generating three-dimensional images” and application number 2023111040410, filed on August 29, 2023, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for generating three-dimensional images. Background Art

[0003] It is expected that diverse three-dimensional (3D) avatars will be generated and used in many application scenarios, including people's daily social interactions, games, videos, and the like. A 3D avatar, also known as a 3D digital avatar, refers to a three-dimensional body that can vividly reflect the visual characteristics of an object. In conventional solutions, 3D avatars are usually created and depicted laboriously by an artist. This is not only laborious, but also difficult to create large-scale and diverse 3D avatars. With the advancement of computer vision technology based on machine learning, generative models are being used to automatically generate 3D data. It is expected that 3D avatars that can be animated can be automatically generated.

[0004] Summary of the Invention

[0005] In a first aspect of the present disclosure, a method for generating a three-dimensional image is provided. The method comprises: generating reference information in a reference space corresponding to a reference form based on input features of a target object, the reference information indicating a reference surface of the target object in the reference form; determining a target form of the target object; and generating three-dimensional image information of the target object in the target form based on the reference information and a transformation relationship between the target form and the reference form.

[0006] In a second aspect of the present disclosure, a device for generating a three-dimensional image is provided. The device includes: a reference information generation module configured to generate reference information in a reference space corresponding to a reference form based on input features of a target object, the reference information indicating a reference surface of the target object in the reference form; a target form determination module configured to determine a target form of the target object; and an image information generation module configured to generate three-dimensional image information of the target object in the target form based on the reference information and a transformation relationship between the target form and the reference form.

[0007] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit. When executed by the at least one processing unit, the instructions cause the device to perform the method of the first aspect.

[0008] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0009] It should be understood that the content described in this summary section is not intended to limit the key features or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0011] FIG1 shows a schematic diagram of an example environment in which embodiments of the present disclosure can be implemented;

[0012] FIG2 shows a schematic block diagram of a system for generating a 3D image according to some embodiments of the present disclosure;

[0013] FIG3 shows a schematic block diagram of a subsystem for generating 3D image information according to some embodiments of the present disclosure;

[0014] FIG4 shows a schematic diagram of 3D image rendering and exemplary training according to some embodiments of the present disclosure;

[0015] FIG5 shows a schematic diagram of an exemplary training of a generator according to some embodiments of the present disclosure;

[0016] FIG6 shows a flowchart of a process for generating a 3D image according to some embodiments of the present disclosure;

[0017] FIG7 shows a block diagram of an apparatus for generating a 3D image according to some embodiments of the present disclosure; and

[0018] FIG8 shows a block diagram of a device capable of implementing various embodiments of the present disclosure. DETAILED DESCRIPTION

[0019] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0020] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0021] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0023] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) must comply with the requirements of relevant laws, regulations and relevant provisions.

[0024] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0025] It should be noted that the titles of any section / subsection provided herein are not limiting. Various embodiments are described throughout this document, and any type of embodiment may be included under any section / subsection. Furthermore, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or in different sections / subsections.

[0026] Herein, unless explicitly stated otherwise, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0027] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0028] As used herein, the term "machine learning model" can learn the association between corresponding inputs and outputs from training data, thereby generating corresponding outputs for a given input after training. Deep learning is a machine learning algorithm that uses multiple layers of processing units to process inputs and provide corresponding outputs. In this article, a "machine learning model" may also be referred to as a "machine learning network" or "network", and these terms are used interchangeably herein.

[0029] As used herein, the term "form" may refer to the shape, pose, or a combination thereof of an object. One form may differ from another in that the shape is different, the pose is different, or both are different.

[0030] Sample Environment

[0031] FIG1 illustrates a block diagram of an example environment 100 capable of implementing various implementations of the present disclosure. In the environment of FIG1 , a system 105 is configured to generate a 3D avatar. System 105 can be implemented on a user device 110 and / or a remote 3D avatar generation device 120. A 3D avatar, also known as a 3D digital avatar, refers to a three-dimensional body or model that can vividly reflect the visual characteristics of an object. Objects that can be three-dimensionally modeled by system 105 may include, but are not limited to, humans (e.g., a human head, half-body, or full-body image), animals, plants, or other static and dynamic objects, and may even include composite objects or scenes. FIG1 illustrates a human being as an example, with 3D avatars 132-1, 132-2, and 132-3 generated by system 105 in different poses, collectively or individually referred to as 3D avatars 132. In some of the following embodiments, humans are used as examples of target objects, but it should be understood that this is merely exemplary and not intended to be limiting. The embodiments described with reference to "humans" are also applicable to other types of objects.

[0032] In some embodiments, system 105 can generate a 3D image based on a user request from user device 110. In some embodiments, if system 105 is running on a remote 3D image generation device 120 rather than locally on user device 110, the user request can be sent to 3D image generation device 120 via network 130. The generated 3D image 132 can be sent to user device 110 via network 130. In some embodiments, 3D image generation device 120 can generate a 3D image based on other triggering events. In some embodiments, 3D image generation device 120 can provide 3D image generation services in response to requests from multiple user devices.

[0033] User device 110 may include any type of mobile, fixed, or portable terminal, including a mobile phone, desktop computer, laptop computer, notebook computer, netbook computer, tablet computer, media computer, multimedia tablet, or any combination thereof, including accessories and peripherals of these devices or any combination thereof. Servers include, but are not limited to, mainframe computers, edge computing nodes, computing devices in cloud environments, etc. 3D image generation device 120 may be any electronic device with computing capabilities, including servers, blade servers, mainframe computers, edge devices, and other appropriate computing devices.

[0034] It should be understood that the components and arrangements of environment 100 shown in FIG1 are merely examples, and that a computing system suitable for implementing the exemplary implementations described herein may include one or more different components, other components, and / or different arrangements. Specific examples of images and graphics are shown in FIG1 and other figures for illustrative purposes only and do not limit the specific implementations of the present disclosure.

[0035] As briefly mentioned above, generative models have been used to automatically generate 3D images. Some solutions are based on generative adversarial networks (GANs). However, these solutions are suitable for modeling relatively simple and rigid objects, but have difficulty modeling dynamic human bodies with articulated motions. Other solutions introduce prior knowledge of the human body into the generative model in order to generate animatable 3D human figures. However, these solutions do not explicitly model the human body surface, and the computational overhead of these solutions is high. Therefore, these solutions have difficulty in achieving high-resolution 3D image rendering.

[0036] The embodiments of the present disclosure propose a solution for 3D image generation. According to various embodiments of the present disclosure, reference information in a reference space corresponding to a reference form is generated based on input features for a target object. The reference information indicates a reference surface of the target object in the reference form. The target form of the target object is determined. Based on the reference information and the transformation relationship between the target form and the reference form, three-dimensional image information of the target object in the target form is generated. In the embodiments of the present disclosure, the surface of the target object in the reference form is first modeled, and then the surface in the reference form is converted to the target form based on the transformation relationship between the reference form and the target form. By modeling the surface in the reference form, on the one hand, the ability to process surface details (for example, wrinkles on clothes, hair, etc.) can be improved, and on the other hand, the image in the desired target form can be generated in a controllable manner. In this way, high-quality 3D image generation in various desired forms can be advantageously achieved.

[0037] Some example implementations of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0038] Example system for 3D character generation

[0039] Figure 2 shows a schematic block diagram of a system for generating 3D characters according to some embodiments of the present disclosure. The system of Figure 2 can be implemented, for example, as system 105 of Figure 1. As shown in Figure 2, system 105 includes at least a 3D character information generation subsystem 210, also referred to as subsystem 210. In some embodiments, system 105 may also include a rendering subsystem 220 for rendering 3D characters. The various components and models in system 105 may be implemented using hardware, software, firmware, or any combination thereof.

[0040] As shown in FIG2 , subsystem 210 takes as input input features 201 for a target object. Input features 201 can be any suitable type of vectorized representation. In some embodiments, input features 201 can be an implicit encoding randomly sampled from a Gaussian distribution. In some embodiments, input features 201 can be a representation optimized based on the desired target 3D image, as described in detail below. In some embodiments, input features 201 can include multiple components for different feature dimensions, such as an input component for geometric features and an input component for texture features. Such an embodiment will be described below with reference to FIG3 .

[0041] The subsystem 210 generates reference information 211 in a reference space corresponding to a reference form based on the input features 201. The reference information 211 indicates the surface of the target object in the reference form, also referred to as the reference surface. For example, the reference space can be a canonical space, in which the object representation (e.g., human body representation) can be regarded as a kind of template. The reference form can have any suitable shape and pose. In particular, such a reference form has as many exposed surfaces as possible to more comprehensively characterize the object. For example, when the target object is a human, the reference form can be a star pose with the arms and legs separated, which is similar to the Chinese character "大". Another example is that the reference form can also be a T pose or an A pose. The embodiments of the present disclosure are not limited in this regard.

[0042] The generated reference information 211 can be used to characterize different dimensional features of the target object in the reference form, such as geometric features and texture features. In some embodiments, the reference information 211 can include reference geometric information in the reference space, which indicates the distance between points in the reference space and the reference surface, for characterizing the geometric features of the target object in the reference form. The reference information 211 can also include reference texture information in the reference space, which indicates the texture corresponding to points in the reference space, for characterizing the texture features of the target object in the reference form.

[0043] Additionally, in some embodiments, the reference information 211 can include reference normal information in the reference space, which indicates the normal at each position on the reference surface. The reference normal information can be represented by, for example, a normal field. Using the reference normal information, complex surfaces can be modeled. In this way, the geometric details of the generated 3D image can be improved. For example, when the target object is a human, details such as wrinkles on the clothes can be generated.

[0044] As shown in FIG. 2, the subsystem 210 also utilizes the target form of the target object. The target form can be represented by any suitable parameters. Exemplarily, when the target object is a human, a Skinned Multi-Person Linear (SMPL) model can be used to represent the target form. The SMPL model is a vertex-based parametric human model that can represent different shapes and poses of the human body. For example, the SMPL parameters p = (θ, β) include shape parameters β and pose parameters θ. The shape parameters β can be used to define the shape of the human body, such as height, weight, body proportion, etc. The pose parameters θ can be used to define the pose of the overall movement of the human body and the relative angles of several joints (such as 24 joints). It should be understood that the SMPL parameters are only exemplary, and other suitable parameters can also be used to represent the form.

[0045] The target morphology can be determined from a variety of suitable input sources. In some embodiments, text describing the target morphology can be received. The text can be processed to determine the target morphology desired by the user and to provide a parameterized representation. In some embodiments, the target morphology can be derived from one or more reference images, which can be still images or videos. For example, morphology recognition can be performed on reference objects in an input image or video, and the target morphology can be determined based on the recognized morphology. For example, the recognized morphology can be parameterized.

[0046] Continuing with reference to FIG. 2 , after acquiring the target form, subsystem 210 can determine the transformation relationship between the target form and the reference form. Based on reference information 211 and the transformation relationship, subsystem 210 can generate 3D image information 212 of the target object in the target form, which is information in the target space corresponding to the target form. 3D image information 212 can represent the target object in the target form in the target space in various suitable ways. For example, 3D image information 212 can be represented by a mesh. In this case, 3D image information 212 can be considered an explicit representation of the target object.

[0047] In some embodiments, system 105 may further include a rendering subsystem 220 for rendering a target object having a target form to more intuitively present the target object. Because 3D image information 212 includes information about various orientations of the target object, the target object can be rendered from various viewing angles. A desired target viewing angle can be determined based on camera parameter information. Thus, rendering subsystem 225 can render an image of the target object having the target form at a target viewing angle based on 3D image information 212, also referred to as a target image.

[0048] For example, based on the 3D image information 212, the rendering subsystem 220 can generate a 3D image of the target object by rendering the 3D image information 212. Then, by changing the viewing angle, an image of the target object at a desired viewing angle can be generated. FIG. 2 exemplarily shows images 226-1, 226-2, and 226-3 corresponding to different viewing angles. In this way, the target object can be presented from a variety of desired viewing angles.

[0049] An example of a system 105 for generating a 3D image has been described above with reference to FIG2 . Generally speaking, system 105 deforms a reference representation of a target object in a reference space into a target shape and pose through volume deformation. In other words, the target object in a target pose is represented in a space corresponding to the target form. This enables the generation of a morphologically controllable 3D image.

[0050] Example generation of 3D image information

[0051] The target object in the target form may include features of different dimensions, such as geometric features and texture features. In some embodiments, the subsystem 210 may include a branch for geometric features (also referred to as a geometry branch) and a branch for texture features (also referred to as a texture branch). To this end, the subsystem 210 may include a generator. By dividing into a geometry branch and a texture branch, the learning of features of different dimensions can be separated, thereby reducing the learning difficulty of the generator.

[0052] An example is described below with reference to Figure 3. As shown in Figure 3, the generator includes at least a geometry mapping network 301, a texture mapping network 302, and a backbone network 303. It should be understood that these networks can be implemented using any suitable structure, and the embodiments of the present disclosure are not limited in this respect.

[0053] As shown in FIG2 , the input features include an input component for geometric features, also referred to as the first input component or geometric component z geo The input features also include an input component for texture features, also known as the second input component or texture component z tex As described above with reference to FIG. 2 , the geometry component and the texture component may be randomly sampled from a Gaussian distribution, or may be optimized based on a desired target image.

[0054] The following are example processing examples for the geometry branch and the texture branch, respectively. For the geometry branch, based on the geometry component, reference geometry information 321 in the reference space can be generated, indicating the distance between a point in the reference space and a reference surface. In the example of FIG3 , reference geometry information 321 is shown as a signed distance field. Each point in the signed distance field is represented by a signed value, where the positive or negative sign indicates whether the point is inside or outside the reference surface, and the magnitude of the value represents the distance from the reference surface. However, it should be understood that the signed distance field is merely an example implementation of reference geometry information and is not intended to be limiting.

[0055] The example process of generating reference geometric information is described with reference to the example of FIG3. The geometric mapping network 301 is based on the input geometric component z geo Generate geometric feature representation w as an intermediate result geo The backbone network 303 can represent w based on geometric features. geo A geometric three-plane feature representation 311 of the geometric branch is generated. The geometric three-plane feature representation 311 includes three 2D feature maps, each corresponding to one of the three planes.

[0056] Compared to other data representations of 3D volumes (such as voxels and point clouds), triplanar feature representations require less data, require less storage space, and enable faster model execution without sacrificing 3D representation. This can significantly reduce resource overhead for model training and application.

[0057] Then, the geometric three-plane feature representation 311 can be converted into a signed distance field in the reference space as reference geometric information. For example, as shown in FIG3 , for any 3D point in the reference space, the geometric features of the point can be queried from the geometric three-plane feature representation 311. Specifically, the point can be projected onto each plane in the three planes and the features on each plane can be aggregated to obtain the geometric features of the point. geo A multi-layer perception (MLP) network with conditions is used to decode the geometric features of the queried point into the directed distance value of the point. It should be understood that the process of generating reference geometric information described here is only exemplary.

[0058] For the texture branch, based on the texture components, reference texture information 323 in the reference space can be generated, indicating the texture corresponding to a point in the reference space. In the example of FIG3 , reference texture information 323 is shown as a texture field. The value of each point in the texture field represents the color of that point. However, it should be understood that the texture field is merely an example implementation of reference geometric information and is not intended to be limiting.

[0059] An example process of generating reference texture information is described with reference to the example of FIG3. The texture mapping network 302 is based on the input texture component z tex Generate texture feature representation w as an intermediate result tex The backbone network 303 can represent w based on texture features. tex A texture three-plane feature representation 312 of the texture branch is generated. Similar to the geometric three-plane feature representation 311, the texture three-plane feature representation 312 includes three 2D feature maps, each corresponding to one of the three planes.

[0060] Then, the texture three-plane feature representation 312 can be converted into a texture field in the reference space as reference texture information 323. For example, as shown in FIG3 , for any 3D point in the reference space, the texture feature of the point can be queried from the texture three-plane feature representation 312. Specifically, the point can be projected onto each plane in the three planes and the features on each plane can be aggregated to obtain the texture feature of the point. tex and geometric feature representation w geo The MLP network with the condition is used to decode the texture feature of the queried point into the color value of the point. In this example, since texture generation is affected by geometric changes, texture feature representation w is used.tex and geometric feature representation w geo Both serve as conditions for the MLP network. It should be understood that the above-described process of generating reference texture information is merely exemplary and not intended to be limiting.

[0061] Continuing with FIG3 , to transform the reference form into the target form, a transformation relationship can be established between the reference form and the target form, that is, a transformation relationship between the reference space and the target space. Based on this transformation relationship, reference geometric information 321 can be converted into target geometric information 331 in the target space. Target geometric information 331 indicates whether a point in the target space is on the target surface, which is the surface of the target object in the target form.

[0062] For ease of description, the morphology of the object is represented by the SMPL model, and it is assumed that the target morphology has a shape β and a posture θ. Accordingly, the target space can be represented as (θ, β), and the target geometry information 331 can be implemented as an SMPL patch M(θ, β). For any point x in the target space, d , the corresponding point x in the reference space can be found through the deformation process c Find the point x on the SMPL patch M(θ,β) d The closest vertex v * Assume that between the reference space and the target space, point x d and its closest vertex v * Subject to the same kinematic changes. Then, we can use the vertex v * The skinning weight of the point x d Transform to point x in the reference space c , as shown below:

[0063] where N j represents the number of joints in SMPL (e.g., 24), Represents vertex v * Relative to the skin weight of the i-th joint, B i (θ, β) is the bone transformation matrix of the i-th joint. For any point x in the target geometry information 331 d , using formula (1) we can find the corresponding point x in the reference space c .

[0064] In this embodiment, the reference shape can be transformed into any desired target shape using the SMPL-guided deformation process, thereby achieving controllable image generation.

[0065] Continuing to describe the processing for the target space, the target geometry information 331 can be represented by a tetrahedral mesh, such as shown in FIG3 . For any vertex x on the tetrahedral mesh, d , use formula (1) to determine its corresponding point x in the reference space c Query point x from reference geometry information 321 c The signed distance value of the tetrahedral mesh vertex x d The directed distance value d(x d ), that is, d(x d )=d(x c ).

[0066] The target geometry information 331 represented by the tetrahedral mesh can then be converted into 3D image information 332 represented by facets (such as triangular facets). For example, the tetrahedral mesh can be converted into a plane using a differentiable marching tetrahedron algorithm. Specifically, if two vertices of an edge in the tetrahedral mesh have directed distance values ​​of different signs, a mesh face can be extracted. Thus, the position information in the facet can represent the position of the surface of the target object in the target form in the target space.

[0067] For the texture branch, a point in the target space located on the target surface is determined. Based on the transformation relationship, similar to formula (1), a reference point corresponding to the point in the reference space can be determined. Based on the texture at the reference point indicated by the reference texture information 323, the texture at the point is determined. Exemplarily, for an embodiment in which 3D image information is represented by a patch, for any point on the patch surface, formula (1) can be used to find the reference point in the reference space corresponding to the point. The color value of the reference point in the texture field can be determined as the color value of the point and stored as part of the patch.

[0068] The above describes an example process of combining the geometry branch and the texture branch to generate 3D image information. It should be understood that the representation of the various information described above is only exemplary and is not intended to limit the scope of the present disclosure.

[0069] In some embodiments, normal information in the reference space can also be constructed to generate finer geometric details, such as hair, clothing wrinkles, etc. Specifically, based on the reference geometric information 321, reference normal information 322 in the reference space can be determined, which indicates the normal at each position on the reference surface. Exemplarily, the reference normal information 322 can be represented by a normal field, where each point on the reference surface has a normal vector. The direction and magnitude of the normal vector represent the orientation and curvature of the surface at that point. For example, the reference normal information can be constructed by calculating the spatial gradient of the signed distance field in the reference space, as shown in the following formula:

[0070] where d(x c ) and n(x c ) are the reference points x in the reference space c The signed distance value and normal vector of .

[0071] After obtaining the reference normal information 322, the reference point x indicated by the reference normal information can be similarly transformed based on the transformation relationship. c The normal at point x is converted to the target surface d For example, the normal of point x on the target surface d The normal vector n(x d ) can be determined by the following formula:

[0072] in Represents vertex v * Relative to the skin weight of the i-th joint, R i (θ,β) is B in formula (1) i The rotation component of (θ,β).

[0073] The exemplary generation process of 3D image information is described above with reference to Figure 3. Thus, a representation of the target object in any desired target form can be obtained.

[0074] Example Rendering

[0075] As mentioned above, the obtained 3D image information can be rendered into a 3D image at a target perspective. For example, a desired target perspective can be determined based on camera parameter information. Based on the 3D image information 332, a target image of a target object having a target form at the target perspective can be rendered. Using the example of 3D image information 332 represented by a mesh, a rendering example will be described with reference to FIG4 .

[0076] Generally speaking, if an implicit neural representation is applied to 3D image information, neural volume rendering is used to generate 3D images. However, this neural volume rendering consumes a lot of computing resources, making it difficult to generate high-resolution images. Using a patch representation can support efficient rasterization-based rendering, thereby generating high-resolution images, such as up to 1024*1024 images.

[0077] Exemplarily, given camera parameters, i.e., a given viewing angle, a rasterizer can be used to project 3D image information 332 (e.g., a patch) into a 2D mask map 401 and a coordinate map (not shown). It should be understood that any suitable rasterizer can be used. Any pixel on the coordinate map can store the 3D coordinates of a corresponding point on the surface of the patch. Then, for each pixel on the coordinate map, the 3D coordinates of the corresponding point can be converted to a reference space to query the color value from the reference texture information in the reference space. In this way, an image 403 at a given viewing angle, such as a grayscale image or an RGB image, can be obtained.

[0078] In some embodiments, if reference normal information in the reference space is generated, a normal map 402 of the target object at a given viewpoint can also be generated. Each pixel in the normal map 402 represents the normal vector at the corresponding point. In such embodiments, thanks to the differentiable surface modeling and rendering techniques described above, the normal map at a given viewpoint can be rendered directly from the normal field in the reference space. This enables adversarial training on the normal map during model training, thereby helping to capture high-frequency geometric details such as hair and wrinkles.

[0079] The above describes rendering for a given perspective. It should be understood that during model inference, the given perspective is the desired target perspective; during model training, the given perspective is the sample perspective.

[0080] The above describes an example generation of 3D image information and an example rendering of a 3D image. In order to more clearly understand the embodiments of the present disclosure, an example application scenario is described below. The user device 110 or the system 105 obtains one or more reference images, such as one or more static images, or frames in a reference video. Exemplarily, the reference image can be input by the user or acquired by an image acquisition device. Morphological recognition is performed on the reference object in the one or more reference images obtained, and the morphology of the identified reference object is used as the target morphology. Then, the 3D image information of the target object in the target morphology can be generated through the example process described above with reference to FIG2 or FIG3. Additionally, the target perspective can also be determined based on the camera perspective in one or more reference images. For example, the camera perspective can be used as the target perspective. In this way, the 3D image information can be rendered according to the target perspective to obtain one or more target images of a 3D image with a target morphology.

[0081] It should be understood that if the reference objects in different reference images have different morphologies (for example, a person in motion in a video), the object morphologies in the rendered target images may also be different. Similarly, if the camera perspectives in different reference images are different, different target images can be rendered from different perspectives. In this example scenario, using the embodiments of the present disclosure, it is possible to conveniently track and reproduce the morphology and / or camera perspective of an object in one or more reference images.

[0082] Example training and input optimization

[0083] The following describes the training of a generator for 3D image information (e.g., geometry mapping network 301, texture mapping network 302, and backbone network 303) with reference to FIG5 . The generator generates sample 3D image information 500 of a sample object in a sample form. It will be appreciated that the generation process for sample 3D image information 500 is similar to that described above for 3D image information 212 and 332, and therefore will not be further elaborated.

[0084] As shown in Figure 5, based on the sample 3D image information 500 and the sample perspective related to the camera parameters, a mask map 501 and an image 503 of the sample object under the sample perspective can be obtained. The determination of the mask map 501 and the image 503 is similar to the determination of the mask map 401 and the image 403 described above, so they are not repeated here. Using the discriminator 511, a discrimination result 521 for the mask map 501 can be determined, which indicates whether the mask map 501 is a real mask map. Using the discriminator 513, a discrimination result 523 for the image 503 can be determined, which indicates whether the image 503 is a real image. At least based on the discrimination result 521 and the discrimination result 523, an adversarial loss 530 can be determined for updating the generator.

[0085] In some embodiments, if reference normal information is generated, a normal map 502 of the sample object from the sample's perspective can be obtained based on the sample 3D image information 500. The determination of normal map 502 is similar to the determination of normal map 402 described above, and therefore will not be further described. Furthermore, a discriminator 512 can be used to determine a judgment result 522 for normal map 502. An adversarial loss is further determined based on judgment results 521, 522, and 523, thereby updating the generator.

[0086] In some embodiments, in addition to the adversarial loss, other losses may be additionally considered to update the generator. For example, in order to make the surface normal vector have a unit 2 norm, the Eikonal loss may be applied to the surface indicated by the generated 3D image information. As shown in the following formula:

[0087] where x iand d(x i ) represent points on the surface and their directed distance values ​​respectively.

[0088] For example, you can use cross entropy loss To regularize the directed distance value in the 3D image information, as shown in the following formula:

[0089] Among them, H, σ and sign are binary cross entropy function, sigmoid function and sign function respectively. is the set of edges whose directed distance values ​​between two vertices have different signs.

[0090] In this case, the total loss function used for the generator is as follows:

[0091] where λ eik and λ ce are the weights of Eikonal loss and cross entropy loss, Denote adversarial loss 530 such as shown in FIG5 .

[0092] In this way, the generators, such as the geometry mapping network 301, the texture mapping network 302, and the backbone network 303, etc., can be trained.

[0093] The above describes an example training of the generator. As briefly mentioned above, in some embodiments, input features 201, such as geometric components and texture components, are optimizable. Specifically, description information for the target 3D image can be obtained. The description information can specify at least a portion of the features of the desired 3D image, such as specifying morphological features, texture features, or a combination thereof. For example, the description information can indicate the generation of a person squatting in a red shirt. In some embodiments, the description information can be text. Alternatively or additionally, the description information can include one or more images.

[0094] Initial 3D image information can be generated using the initial features for the target object. For example, the initial features can be features randomly sampled from a Gaussian distribution, or preset features. The process of generating the initial 3D image information is similar to the process described above for the 3D image information 212 and 332, so it will not be repeated here. Based on the description information and the initial 3D image information, the difference between the 3D image corresponding to the initial 3D image information and the target 3D image can be determined. Based on the difference, the initial features can be optimized. Specifically, the initial features can be updated based on the difference until a preset condition is met, such as the difference is less than a threshold or the number of iterative updates reaches a threshold number. In this way, the updated initial features can be determined as the input features 201.

[0095] Depending on the specific form of the description information, the difference can be determined in a corresponding manner. For example, in the case where the description information is text, the initial 3D image information can be used for rendering to obtain a corresponding image, such as described in reference image 403. Then, the image-text matching degree between the obtained image and the text as the description information can be calculated, and the initial features can be optimized based on the image-text matching degree. Alternatively or additionally, the similarity between the text features of the text as the description information and the image features of the obtained image can be calculated, and the initial features can be optimized based on the similarity. In the case where the description information is an image, the initial features can be optimized based on the similarity between the image and the image obtained by rendering.

[0096] It should be understood that when the input features 201 include a geometric component and a texture component, both components can be optimized simultaneously, or one component can be optimized first and then the other. In this embodiment, the input features are optimized based on the desired target 3D image. In this way, the accuracy of the generated 3D image can be further improved.

[0097] Example Process

[0098] FIG6 shows a flow chart of a process 600 for generating a 3D image according to some embodiments of the present disclosure. The process 600 may be implemented at the system 105 of FIG2 .

[0099] At block 610 , the system 105 generates reference information in a reference space corresponding to a reference form based on input features of the target object. The reference information indicates a reference surface of the target object in the reference form.

[0100] In some embodiments, the input features are determined as follows: obtaining descriptive information for the target three-dimensional image; generating initial three-dimensional image information based on the initial features for the target object; determining the difference between the three-dimensional image corresponding to the initial three-dimensional image information and the target three-dimensional image based on the descriptive information and the initial three-dimensional image information; based on the difference, updating the initial features until a preset condition is met; and determining the updated initial features as the input features.

[0101] At block 620 , the system 105 determines a target morphology of the target object. In some embodiments, determining the target morphology of the target object includes: identifying a morphology of a reference object in a reference image; and determining the target morphology based on the identified morphology.

[0102] At block 630 , the system 105 generates three-dimensional image information of the target object in the target form based on the reference information and the transformation relationship between the target form and the reference form. In some embodiments, the three-dimensional image information is represented by a patch.

[0103] In some embodiments, the input features include a first input component for geometric features and a second input component for texture features. In some embodiments, generating reference information includes: generating reference geometric information in a reference space based on the first input component, the reference geometric information indicating a distance between a point in the reference space and a reference surface; and generating reference texture information in the reference space based on the second input component, the reference texture information indicating a texture corresponding to the point in the reference space.

[0104] In some embodiments, generating three-dimensional image information includes: based on a transformation relationship, converting reference geometric information into target geometric information in a target space corresponding to the target form, the target geometric information indicating whether a point in the target space is on a target surface, the target surface being the surface of the target object in the target form; based on the target geometric information, determining the position of the target surface in the target space as part of the three-dimensional image information; and based on the reference texture information, position and transformation relationship, determining the texture of the target surface as part of the three-dimensional image information.

[0105] In some embodiments, determining the texture of the target surface includes: determining a first point in the target space located on the target surface based on the position; determining a second point in the reference space corresponding to the first point based on the transformation relationship; and determining the texture at the first point based on the texture at the second point indicated by the reference texture information.

[0106] In some embodiments, process 600 also includes: determining reference normal information in a reference space based on reference geometric information, the reference normal information indicating the normal at each position on the reference surface; and based on a transformation relationship, converting the normal at the second point indicated by the reference normal information into the normal at the first point as part of the three-dimensional image information.

[0107] In some embodiments, generating reference geometric information includes: using a trained generator to generate a set of geometric feature maps corresponding to three planes respectively from the first input component to characterize the geometric features of the target object in the reference form; and for a point in the reference space, based on the features corresponding to the point in a set of geometric feature maps respectively, determining the distance between the point and the reference surface.

[0108] In some embodiments, generating reference texture information includes: using a trained generator to generate a set of texture feature maps corresponding to three planes respectively from the second input component to characterize the texture features of the target object in the reference form; and for a point in the reference space, based on the features corresponding to the point respectively in a set of texture feature maps, determining the texture corresponding to the point.

[0109] In some embodiments, the reference information is determined by a generator based on the input features. In some embodiments, the training step of the generator includes: generating, using the generator, sample three-dimensional image information of the sample object in the sample morphology; obtaining, based on the sample three-dimensional image information, a mask map and an image of the sample object from the sample perspective; determining, using a first discriminator, a first discrimination result for the mask map; determining, using a second discriminator, a second discrimination result for the image; and updating the generator based on at least the first discrimination result and the second discrimination result.

[0110] In some embodiments, the sample 3D image information also includes normal information for each position on the surface of the sample object in the sample form. In some embodiments, training the generator further includes: obtaining a normal map of the sample object from the sample's perspective based on the sample 3D image information; determining a third discrimination result for the normal map using a third discriminator; and further updating the generator based on the third discrimination result.

[0111] Example devices and equipment

[0112] FIG7 shows a schematic block diagram of an apparatus 700 for generating a 3D image according to certain embodiments of the present disclosure. Apparatus 700 may be implemented as or included in implementation system 105, user device 110, or 3D image generation device 120. Each module / component in apparatus 700 may be implemented using hardware, software, firmware, or any combination thereof.

[0113] As shown, apparatus 700 includes a reference information generation module 710 configured to generate reference information in a reference space corresponding to a reference form based on input features of a target object, wherein the reference information indicates a reference surface of the target object in the reference form. Apparatus 700 also includes a target form determination module 720 configured to determine a target form of the target object. Apparatus 700 also includes an image information generation module 730 configured to generate three-dimensional image information of the target object in the target form based on the reference information and a transformation relationship between the target form and the reference form.

[0114] In some embodiments, the input features include a first input component for geometric features and a second input component for texture features, and the reference information generation module 710 includes: a geometric reference information generation module, configured to generate reference geometric information in a reference space based on the first input component, the reference geometric information indicating the distance between a point in the reference space and a reference surface; and a reference texture information generation module, configured to generate reference texture information in the reference space based on the second input component, the reference texture information indicating the texture corresponding to the point in the reference space.

[0115] In some embodiments, the image information generation module 730 includes: an information conversion module, configured to convert the reference geometric information into target geometric information in a target space corresponding to the target form based on a transformation relationship, the target geometric information indicating whether a point in the target space is on a target surface, and the target surface is the surface of the target object in the target form; a position determination module, configured to determine the position of the target surface in the target space based on the target geometric information as part of the three-dimensional image information; and a texture determination module, configured to determine the texture of the target surface based on the reference texture information, position and transformation relationship as part of the three-dimensional image information.

[0116] In some embodiments, the texture determination module is further configured to: determine a first point in the target space located on the target surface based on the position; determine a second point in the reference space corresponding to the first point based on the transformation relationship; and determine the texture at the first point based on the texture at the second point indicated by the reference texture information.

[0117] In some embodiments, the device 700 also includes: a reference normal information determination module, configured to determine reference normal information in a reference space based on reference geometric information, the reference normal information indicating the normal at each position on the reference surface; and a normal conversion module, configured to convert the normal at the second point indicated by the reference normal information into the normal at the first point based on a transformation relationship, as part of the three-dimensional image information.

[0118] In some embodiments, the reference geometric information generation module is further configured to: use a trained generator to generate a set of geometric feature maps corresponding to three planes respectively from the first input component to characterize the geometric features of the target object in the reference form; and for a point in the reference space, determine the distance between the point and the reference surface based on the features corresponding to the point in a set of geometric feature maps respectively.

[0119] In some embodiments, the reference texture information generation module is further configured to: use a trained generator to generate a set of texture feature maps corresponding to the three planes respectively from the second input component to characterize the texture features of the target object in the reference form; and for a point in the reference space, determine the texture corresponding to the point based on the features corresponding to the point in a set of texture feature maps respectively.

[0120] In some embodiments, the device 700 also includes the following modules to determine the input features: a description information acquisition module, configured to obtain description information for the target three-dimensional image; an initial three-dimensional image information generation module, configured to generate initial three-dimensional image information based on the initial features of the target object; a difference determination module, configured to determine the difference between the three-dimensional image corresponding to the initial three-dimensional image information and the target three-dimensional image based on the description information and the initial three-dimensional image information; an initial feature update module, configured to update the initial features based on the differences until a preset condition is met; and an input feature determination module, configured to determine the updated initial features as the input features.

[0121] In some embodiments, the reference information is determined based on the input features using a generator, and the device 700 also includes a sample three-dimensional image information generation module, which is configured to use the generator to generate sample three-dimensional image information of the sample object in the sample form; a mask map and image acquisition module, which is configured to obtain the mask map and image of the sample object under the sample perspective based on the sample three-dimensional image information; a first result determination module, which is configured to use the first discriminator to determine the first discrimination result for the mask map; a second result determination module, which is configured to use the second discriminator to determine the second discrimination result for the image; and a generator update module, which is configured to update the generator based on at least the first discrimination result and the second discrimination result.

[0122] In some embodiments, the sample three-dimensional image information also includes normal information of each position on the surface of the sample object under the sample form, and the device 700 also includes: a normal map acquisition module, configured to obtain a normal map of the sample object under the sample perspective based on the sample three-dimensional image information; a third result determination module, configured to use a third discriminator to determine a third discrimination result for the normal map; and a generator update module configured to further update the generator based on the third discrimination result.

[0123] In some embodiments, the target morphology determination module 720 is further configured to: identify the morphology of the reference object in the reference image; and determine the target morphology based on the identified morphology.

[0124] In some embodiments, the device 700 also includes: a target perspective determination module, configured to determine the target perspective based on camera parameter information; and a target image rendering module, configured to render a target image of a target object with a target form under a target perspective based on three-dimensional image information.

[0125] In some embodiments, the three-dimensional image information is represented by patches.

[0126] FIG8 illustrates a block diagram of an electronic device 800 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the electronic device 800 shown in FIG8 is merely exemplary and should not be construed as limiting the functionality and scope of the embodiments described herein. The electronic device 800 shown in FIG8 may be used to implement the system 105, the user device 110, or the 3D image generation device 120 of FIG1 .

[0127] As shown in FIG8 , electronic device 800 is a general-purpose electronic device. Components of electronic device 800 may include, but are not limited to, one or more processors or processing units 810, memory 820, storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. Processing unit 810 may be a real or virtual processor and is capable of performing various processes according to programs stored in memory 820. In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to enhance the parallel processing capabilities of electronic device 800.

[0128] The electronic device 800 typically includes a plurality of computer storage media. Such media can be any accessible media that can be obtained by the electronic device 800, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 820 can be a volatile memory (e.g., registers, cache, random access memory (RAM)), a non-volatile memory (e.g., read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 830 can be a removable or non-removable medium and can include a machine-readable medium, such as a flash drive, a disk, or any other medium that can be used to store information and / or data and can be accessed within the electronic device 800.

[0129] The electronic device 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Although not shown in FIG8 , a disk drive for reading from or writing to a removable, non-volatile disk (e.g., a “floppy disk”) and an optical drive for reading from or writing to a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. The memory 820 may include a computer program product 825 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

[0130] The communication unit 840 enables communication with other electronic devices via a communication medium. Additionally, the functions of the components of the electronic device 800 can be implemented in a single computing cluster or multiple computing machines that can communicate via a communication connection. Thus, the electronic device 800 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0131] The input device 850 may be one or more input devices, such as a mouse, keyboard, or trackball. The output device 860 may be one or more output devices, such as a display, a speaker, or a printer. The electronic device 800 may also communicate with one or more external devices (not shown) via the communication unit 840 as needed, such as a storage device, a display device, or the like, with one or more devices that allow a user to interact with the electronic device 800, or with any device that allows the electronic device 800 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0132] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0133] Various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatuses, devices, and computer program products implemented according to the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer-readable program instructions.

[0134] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, such that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device is generated that implements the functions / actions specified in one or more blocks in the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium, where these instructions cause the computer, programmable data processing device, and / or other device to operate in a specific manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing various aspects of the functions / actions specified in one or more blocks in the flowchart and / or block diagram.

[0135] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0136] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple implementations of the present disclosure. In this regard, each box in the flow chart or block diagram can represent a part for a module, program segment or instruction, and a part for a module, program segment or instruction comprises one or more executable instructions for realizing the logical function of the specification. In some alternative implementations, the functions marked in the box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0137] While various implementations of the present disclosure have been described above, the foregoing description is intended to be illustrative, not exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described implementations. The terminology used herein is selected to best explain the principles of the implementations, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the various implementations disclosed herein.

Claims

1. A method for generating a three-dimensional image, comprising: generating reference information in a reference space corresponding to a reference form based on input features for the target object, the reference information indicating a reference surface of the target object in the reference form; determining a target morphology of the target object; as well as Based on the reference information and the transformation relationship between the target form and the reference form, three-dimensional image information of the target object in the target form is generated.

2. The method according to claim 1, wherein the input feature comprises a first input component for a geometric feature and a second input component for a texture feature, and generating the reference information comprises: generating reference geometric information in the reference space based on the first input component, the reference geometric information indicating a distance between a point in the reference space and the reference surface; as well as Based on the second input component, reference texture information in the reference space is generated, the reference texture information indicating a texture corresponding to a point in the reference space.

3. The method according to claim 2, wherein generating the three-dimensional image information comprises: Based on the transformation relationship, converting the reference geometric information into target geometric information in a target space corresponding to the target form, the target geometric information indicating whether a point in the target space is on a target surface, the target surface being a surface of the target object in the target form; Based on the target geometric information, determine the position of the target surface in the target space as a part of the three-dimensional image information; as well as Based on the reference texture information, the position and the transformation relationship, the texture of the target surface is determined as a part of the three-dimensional image information.

4. The method of claim 3, wherein determining the texture of the target surface comprises: Based on the position, determining a first point in the target space located on the target surface; Based on the transformation relationship, determining a second point in the reference space corresponding to the first point; as well as The texture at the first point is determined based on the texture at the second point indicated by the reference texture information.

5. The method according to claim 4, further comprising: Based on the reference geometric information, determine reference normal information in the reference space, the reference normal information indicating a normal at each position on the reference surface; as well as Based on the transformation relationship, the normal at the second point indicated by the reference normal information is converted into the normal at the first point as a part of the three-dimensional image information.

6. The method according to claim 2, wherein generating the reference geometric information comprises: Using the trained generator, a set of geometric feature maps corresponding to three planes are generated from the first input component to characterize the geometric features of the target object in the reference form; as well as For a point in the reference space, based on the features in the set of geometric feature maps respectively corresponding to the point, the distance between the point and the reference surface is determined.

7. The method according to claim 6, wherein generating the reference texture information comprises: Using the trained generator, a set of texture feature maps corresponding to the three planes are generated from the second input component to characterize the texture features of the target object in the reference form; as well as For a point in the reference space, a texture corresponding to the point is determined based on features in the set of texture feature maps that respectively correspond to the point.

8. The method according to claim 1, wherein the input feature is determined by: Obtaining description information for the target three-dimensional image; Based on the initial features of the target object, generating initial three-dimensional image information; Based on the description information and the initial three-dimensional image information, determining a difference between the three-dimensional image corresponding to the initial three-dimensional image information and the target three-dimensional image; Based on the difference, updating the initial feature until a preset condition is satisfied; and The updated initial features are determined as the input features.

9. The method according to claim 1, wherein the reference information is determined using a generator based on the input features, and the training step of the generator comprises: Using the generator, generating sample three-dimensional image information of the sample object in the sample form; Based on the sample three-dimensional image information, a mask map and an image of the sample object under the sample viewing angle are obtained; Determine a first discrimination result for the mask image using a first discriminator; Determine a second discrimination result for the image using a second discriminator; as well as The generator is updated based on at least the first discrimination result and the second discrimination result.

10. The method according to claim 9, wherein the sample 3D image information further includes normal information of each position of the sample object on the surface of the sample form, and the training of the generator further includes: Based on the three-dimensional image information of the sample, obtaining a normal map of the sample object at the sample viewing angle; Determine a third discrimination result for the normal map by using a third discriminator; as well as The generator is further updated based on the third discrimination result.

11. The method according to claim 1, wherein determining the target shape of the target object comprises: identifying a morphology of a reference object in a reference image; as well as Based on the identified morphology, the target morphology is determined.

12. The method according to claim 1, further comprising: Determine the target viewing angle based on the camera parameter information; as well as Based on the three-dimensional image information, a target image of the target object having the target shape at the target viewing angle is rendered.

13. The method according to claim 1, wherein the three-dimensional image information is represented by a patch.

14. A device for generating a three-dimensional image, comprising: A reference information generation module, configured to generate reference information in a reference space corresponding to a reference form based on input features of the target object, wherein the reference information indicates a reference surface of the target object in the reference form; a target morphology determination module, configured to determine the target morphology of the target object; as well as The image information generating module is configured to generate three-dimensional image information of the target object in the target form based on the reference information and the transformation relationship between the target form and the reference form.

15. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 13 when executed by the at least one processing unit.

16. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 13.

Images