Method, device and equipment for three-dimensional image generation and storage medium

Abstract

The embodiment of the invention provides a three-dimensional image generation method and device, equipment and a storage medium. The method comprises the steps that reference information in a reference space corresponding to a reference form is generated based on input features of a target object, and the reference information indicates a reference surface of the target object in the reference form; determining a target form of the target object; and generating three-dimensional image information of the target object in the target form based on the reference information and the transformation relation between the target form and the reference form. In this way, high-quality three-dimensional image generation in various desired forms can be advantageously achieved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, devices, apparatuses, and computer-readable storage media for generating three-dimensional images. Background Art

[0002] It is expected to generate and use a variety of three-dimensional (3D) avatars in many application scenarios, including people's daily social interactions, games, videos, etc. 3D avatars, also known as 3D digital avatars, refer to three-dimensional bodies that can vividly reflect the visual characteristics of an object. In conventional solutions, 3D avatars are usually created and portrayed by painters with great effort. This is not only laborious, but also difficult to create large-scale and diverse 3D avatars. With the advancement of computer vision technology based on machine learning, generative models are beginning to be used to automatically generate 3D data. It is expected that animated 3D avatars can be automatically generated. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for generating a three-dimensional image is provided. The method comprises: generating reference information in a reference space corresponding to a reference form based on input features of a target object, the reference information indicating a reference surface of the target object in the reference form; determining a target form of the target object; and generating three-dimensional image information of the target object in the target form based on the reference information and a transformation relationship between the target form and the reference form.

[0004] In a second aspect of the present disclosure, a device for generating a three-dimensional image is provided. The device includes: a reference information generation module configured to generate reference information in a reference space corresponding to a reference form based on input features of a target object, the reference information indicating a reference surface of the target object in the reference form; a target form determination module configured to determine a target form of the target object; and an image information generation module configured to generate three-dimensional image information of the target object in the target form based on the reference information and a transformation relationship between the target form and the reference form.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0010] Figure 2 A schematic block diagram of a system for generating a 3D image according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic block diagram of a subsystem for generating 3D image information according to some embodiments of the present disclosure is shown;

[0012] Figure 4 A schematic diagram showing 3D image rendering and exemplary training according to some embodiments of the present disclosure is shown;

[0013] Figure 5 A schematic diagram showing an exemplary training of a generator according to some embodiments of the present disclosure;

[0014] Figure 6 A flowchart showing a process for 3D character generation according to some embodiments of the present disclosure is shown;

[0015] Figure 7 A block diagram showing an apparatus for 3D image generation according to some embodiments of the present disclosure; and

[0016] Figure 8 A block diagram of a device capable of implementing various embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0017] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0018] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0019] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0020] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0021] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0022] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0023] It should be noted that the titles of any sections / subsections provided herein are not restrictive. Various embodiments are described throughout this article, and any type of embodiment may be included under any section / subsection. In addition, the embodiments described in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or different sections / subsections.

[0024] Herein, unless explicitly stated, executing a step “in response to A” does not mean executing the step immediately after “A” but may include one or more intermediate steps.

[0025] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0026] As used herein, the term "machine learning model" can learn the association between corresponding inputs and outputs from training data, so that after training is completed, the corresponding output can be generated for a given input. Machine deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. In this article, "machine learning model" may also be referred to as "machine learning network" or "network", and these terms are used interchangeably in this article.

[0027] As used herein, the term "modality" may be the shape, pose, or a combination thereof of an object. One modality may differ from another in that the shape is different or the pose is different or both are different.

[0028] Example Environment

[0029] Figure 1 A block diagram of an example environment 100 is shown in which various implementations of the present disclosure can be implemented. Figure 1 In the environment, the system 105 is configured to generate a 3D image. The system 105 can be implemented in the user device 110 and / or the remote 3D image generation device 120. A 3D image, also known as a 3D digital image, refers to a three-dimensional body or a three-dimensional model that can vividly reflect the visual characteristics of an object. Objects that can be three-dimensionally modeled by the system 105 may include, but are not limited to, people (e.g., a person's head, half-body or full-body image), animals, plants, or other static and dynamic objects, and may even include composite objects or scenes, etc. Figure 1 In the embodiment, a person is taken as an example, and 3D images 132-1, 132-2, and 132-3 with different postures generated by the system 105 are provided, which are also collectively or individually referred to as 3D images 132. In some embodiments below, a person is used as an example of a target object, but it should be understood that this is only exemplary and not intended to be limiting. The embodiments described with reference to "a person" are applicable to other types of objects.

[0030] In some embodiments, the system 105 may generate a 3D image based on a user request from the user device 110. In some embodiments, if the system 105 is run on a remote 3D image generation device 120 instead of on the user device 110, the user request may be sent to the 3D image generation device 120 via the network 130. The generated 3D image 132 may be sent to the user device 110 via the network 130. In some embodiments, the 3D image generation device 120 may generate a 3D image based on other triggering events. In some embodiments, the 3D image generation device 120 may provide 3D image generation services in response to requests from multiple user devices.

[0031] The user device 110 may include any type of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof. The server includes but is not limited to a mainframe, an edge computing node, a computing device in a cloud environment, and the like. The 3D image generation device 120 may be any electronic device with computing capabilities, including a server, a blade server, a mainframe, an edge device, and other appropriate computing devices.

[0032] It should be understood that Figure 1 The components and arrangements in the environment 100 shown are merely examples, and a computing system suitable for implementing the exemplary implementations described in the present disclosure may include one or more different components, other components, and / or different arrangements. Figure 1 Some specific examples of images and pictures are shown in the figures and other drawings, but this is only for illustrative purposes and does not bring any limitation to the specific implementation of the present disclosure.

[0033] As briefly mentioned above, generative models have been used for the automatic generation of 3D images. Some solutions are based on generative adversarial networks (GANs). However, these solutions are suitable for modeling relatively simple and rigid objects, but it is difficult to model dynamic human bodies with hinged motions. In order to generate animated 3D human images, some other solutions introduce human body prior knowledge into the generative models. However, these solutions do not explicitly model the human body surface, and the computational overhead of these solutions is large. Therefore, these solutions are difficult to achieve high-resolution 3D image rendering.

[0034] The embodiments of the present disclosure propose a scheme for 3D image generation. According to various embodiments of the present disclosure, reference information in a reference space corresponding to a reference form is generated based on input features for a target object. The reference information indicates a reference surface of the target object in the reference form. The target form of the target object is determined. Based on the reference information and the transformation relationship between the target form and the reference form, three-dimensional image information of the target object in the target form is generated. In the embodiments of the present disclosure, the surface of the target object in the reference form is first modeled, and then the surface in the reference form is converted to the target form based on the transformation relationship between the reference form and the target form. By modeling the surface in the reference form, on the one hand, the ability to process surface details (e.g., clothing wrinkles, hair, etc.) can be improved, and on the other hand, the image in the desired target form can be generated in a controllable manner. In this way, high-quality 3D image generation in various desired forms can be advantageously achieved.

[0035] Some example implementations of the present disclosure will be described in more detail below with reference to the accompanying drawings.

[0036] Example system for 3D character generation

[0037] Figure 2 A schematic block diagram of a system for generating a 3D image according to some embodiments of the present disclosure is shown. Figure 2 A system can be implemented as Figure 1 System 105. Figure 2 As shown, the system 105 includes at least a 3D image information generation subsystem 210, which is also referred to as subsystem 210. In some embodiments, the system 105 may also include a rendering subsystem 220 for rendering a 3D image. The various components and models in the system 105 may be implemented by hardware, software, firmware, or any combination thereof.

[0038] like Figure 2 As shown, the subsystem 210 takes input features 201 for the target object as input. The input features 201 can be any suitable type of vectorized representation. In some embodiments, the input features 201 can be an implicit encoding randomly sampled from a Gaussian distribution. In some embodiments, the input features 201 can be a representation optimized based on the desired target 3D image, as described in detail below. In some embodiments, the input features 201 can include multiple components for different feature dimensions, such as an input component for geometric features and an input component for texture features. Figure 3 Such an embodiment is described.

[0039] The subsystem 210 generates reference information 211 in a reference space corresponding to a reference form based on the input feature 201. The reference information 211 indicates the surface of the target object in the reference form, also referred to as the reference surface. For example, the reference space can be a canonical space, in which the object representation (e.g., human body representation) can be regarded as a kind of template. The reference form can have any suitable shape and pose. In particular, such a reference form has as many exposed surfaces as possible to more comprehensively characterize the object. For example, when the target object is a human, the reference form can be a star pose with the arms and legs separated, which is similar to the Chinese character '大'. Another example is that the reference form can also be a T pose or an A pose. The embodiments of the present disclosure are not limited in this regard.

[0040] The generated reference information 211 can be used to characterize different dimensional features of the target object in the reference form, such as geometric features and texture features. In some embodiments, the reference information 211 can include reference geometric information in the reference space, which indicates the distance between a point in the reference space and the reference surface, for characterizing the geometric features of the target object in the reference form. The reference information 211 can also include reference texture information in the reference space, which indicates the texture corresponding to a point in the reference space, for characterizing the texture features of the target object in the reference form.

[0041] Additionally, in some embodiments, the reference information 211 can include reference normal information in the reference space, which indicates the normal at each position on the reference surface. The reference normal information can be represented by a normal field, for example. Using the reference normal information, complex surfaces can be modeled. In this way, the geometric details of the generated 3D image can be improved. For example, when the target object is a human, details such as wrinkles on the clothes can be generated.

[0042] As Figure 2 shown, the subsystem 210 also utilizes the target form of the target object. The target form can be represented by any suitable parameters. Exemplarily, when the target object is a human, the Skinned Multi-Person Linear (SMPL) model can be used to represent the target form. The SMPL model is a vertex-based parametric human model that can represent different shapes and poses of the human body. For example, the SMPL parameters p = (θ, β) include the shape parameter β and the pose parameter θ. The shape parameter β can be used to define the shape of the human body, such as height, weight, and head-to-body ratio. The pose parameter θ can be used to define the overall movement pose of the human body and the relative angles of several joints (such as 24). It should be understood that the SMPL parameters are only exemplary, and other suitable parameters can also be used to represent the form.

[0043] The target morphology may be determined from a variety of suitable input sources. In some embodiments, text describing the target morphology may be received. The target morphology desired by the user may be determined by processing the text and represented parameterically. In some embodiments, the target morphology may be from one or more reference images, which may be static images or videos. For example, morphology recognition may be performed on a reference object in an input image or video, and the target morphology may be determined based on the recognized morphology. For example, the recognized morphology may be represented parameterically.

[0044] Continue to refer Figure 2 . After acquiring the target form, the subsystem 210 can determine the transformation relationship between the target form and the reference form. Based on the reference information 211 and the transformation relationship, the subsystem 210 can generate 3D image information 212 of the target object in the target form, which is information in the target space corresponding to the target form. The 3D image information 212 can represent the target object in the target form in the target space in various suitable ways. For example, the 3D image information 212 can be represented by a mesh. In this case, the 3D image information 212 can be regarded as an explicit representation of the target object.

[0045] In some embodiments, the system 105 may further include a rendering subsystem 220 for rendering a target object having a target form so as to present the target object more intuitively. Since the 3D image information 212 includes information about each orientation of the target object, the target object can be rendered from various viewing angles. The desired target viewing angle can be determined based on the camera parameter information. In this way, the rendering subsystem 225 can render an image of the target object having a target form at a target viewing angle based on the 3D image information 212, also referred to as a target image.

[0046] For example, based on the 3D image information 212, the rendering subsystem 220 can generate a 3D image of the target object by rendering the 3D image information 212. Then, by changing the viewing angle, an image of the target object under the target viewing angle can be generated. Figure 2 In FIG. 2 , images 226 - 1 , 226 - 2 , and 226 - 3 corresponding to different viewing angles are exemplarily shown. In this way, the target object can be presented from various desired viewing angles.

[0047] References Figure 2 An example of a system 105 for 3D image generation is described. In general, the system 105 deforms a reference representation of a target object in a reference space into a target shape and posture by means of body deformation. That is, the target object in a target posture is represented in a space corresponding to the target morphology. Thus, morphology-controllable 3D image generation can be achieved.

[0048] Example generation of 3D image information

[0049] The target object in the target form may include features of different dimensions, such as geometric features and texture features. In some embodiments, the subsystem 210 may include a branch for geometric features (also referred to as a geometry branch) and a branch for texture features (also referred to as a texture branch). To this end, the subsystem 210 may include a generator. By dividing into a geometry branch and a texture branch, the learning of features of different dimensions can be separated, thereby reducing the learning difficulty of the generator.

[0050] Reference below Figure 3 Describe an example. Figure 3 As shown, the generator includes at least a geometry mapping network 301, a texture mapping network 302, and a backbone network 303. It should be understood that any suitable structure may be used to implement these networks, and the embodiments of the present disclosure are not limited in this respect.

[0051] like Figure 2 As shown, the input features include an input component for geometric features, also called the first input component or geometric component z geo The input features also include an input component for texture features, also called a second input component or texture component z tex As mentioned above Figure 2 As described, the geometry component and the texture component may be randomly sampled from a Gaussian distribution, or may be optimized based on the desired target image.

[0052] The following are example processing of the geometry branch and the texture branch. For the geometry branch, based on the geometry component, reference geometry information 321 in the reference space can be generated, which indicates the distance between a point in the reference space and the reference surface. Figure 3 In the example of , the reference geometric information 321 is shown as a signed distance field. Each point in the signed distance field is represented by a signed numerical value, where the positive or negative sign indicates whether the point is inside or outside the reference surface, and the magnitude of the numerical value indicates the distance from the reference surface. However, it should be understood that the signed distance field is only an example implementation of the reference geometric information, and is not intended to be limiting.

[0053] refer to Figure 3 The example of FIG. 300 describes an example process of generating reference geometric information. The geometric mapping network 301 is based on the input geometric component z geo Generate a geometric feature representation w as an intermediate result geo The backbone network 303 can represent w based on geometric features. geo A geometric three-plane feature representation 311 of the geometric branch is generated. The geometric three-plane feature representation 311 includes three 2D feature maps, each of which corresponds to one of the three planes.

[0054] Compared with other data representation forms of three-dimensional bodies (such as voxels, point clouds, etc.), the feature representation in the form of three planes has less data, requires less storage space without sacrificing three-dimensional expression, and can achieve faster model running speed. This can greatly save resource overhead for model training and application.

[0055] Then, the geometric three-plane feature representation 311 can be converted into a signed distance field in a reference space as reference geometric information. Figure 3 As shown in FIG. 3 , for any 3D point in the reference space, the geometric features of the point can be queried from the geometric three-plane feature representation 311. Specifically, the point can be projected onto each plane in the three planes and the features on each plane can be aggregated to obtain the geometric features of the point. geo A multi-layer perception (MLP) network with conditions is used to decode the geometric features of the queried point into a directed distance value of the point. It should be understood that the process of generating reference geometric information described herein is only exemplary.

[0056] For the texture branch, based on the texture component, reference texture information 323 in the reference space may be generated, which indicates the texture corresponding to the point in the reference space. Figure 3 In the example of FIG. 3 , the reference texture information 323 is shown as a texture field. The value of each point in the texture field represents the color of the point. However, it should be understood that the texture field is only an example implementation of the reference geometric information and is not intended to be limiting.

[0057] refer to Figure 3 The example of FIG. 300 describes an example process of generating reference texture information. The texture mapping network 302 generates reference texture information based on the input texture component z tex Generate texture feature representation w as an intermediate result tex The backbone network 303 can represent w based on texture features. tex A texture three-plane feature representation 312 of the texture branch is generated. Similar to the geometric three-plane feature representation 311, the texture three-plane feature representation 312 includes three 2D feature maps, each corresponding to one of the three planes.

[0058] Then, the texture three-plane feature representation 312 can be converted into a texture field in a reference space as reference texture information 323. For example, Figure 3 As shown in FIG. 3 , for any 3D point in the reference space, the texture feature of the point can be queried from the texture three-plane feature representation 312. Specifically, the point can be projected onto each plane in the three planes and the features on each plane can be aggregated to obtain the texture feature of the point. tex and geometric features w geoThe MLP network with the condition as the condition is used to decode the texture feature of the queried point into the color value of the point. In this example, since texture generation is affected by geometric changes, texture features are used to represent w tex and geometric features w geo Both serve as conditions for the MLP network. It should be understood that the process of generating reference texture information described above is only exemplary and is not intended to be limiting.

[0059] continue Figure 3 In order to transform the reference form into the target form, a transformation relationship between the reference form and the target form can be established, that is, a transformation relationship between the reference space and the target space. Based on the transformation relationship, the reference geometric information 321 can be converted into target geometric information 331 in the target space. The target geometric information 331 indicates whether a point in the target space is on the target surface, which is the surface of the target object in the target form.

[0060] For ease of description, the morphology of the object is represented by the SMPL model, and it is assumed that the target morphology has a shape β and a posture θ. Accordingly, the target space can be represented as (θ, β), and the target geometry information 331 can be implemented as an SMPL patch M(θ, β). For any point x in the target space d , the corresponding point x in the reference space can be found through the deformation process c . Find the point x on the SMPL patch M(θ,β) d The closest vertex v * Assume that between the reference space and the target space, point x d and its closest vertex v * undergoes the same kinematic changes. Then, we can use the vertex v * The skinning weight of point x d Transform to point x in reference space c , as shown below:

[0061]

[0062] Where N j represents the number of joints in SMPL (e.g., 24), Represents vertex v * Relative to the skin weight of the ith joint, B i (θ, β) is the bone transformation matrix of the i-th joint. For any point x in the target geometry information 331 d , using formula (1) we can find the corresponding point x in the reference space c .

[0063] In such an embodiment, the reference shape can be transformed into any desired target shape using the SMPL-guided deformation process, thereby achieving controllable image generation.

[0064] The processing for the target space is described below. The target geometry information 331 can be represented by a tetrahedral mesh, such as Figure 3 For any vertex x on the tetrahedral mesh d , use formula (1) to determine its corresponding point x in the reference space c Query point x from reference geometry information 321 c The signed distance value of the tetrahedral mesh vertex x d The directed distance value d(x d ), that is, d(x d )=d(x c ).

[0065] Then, the target geometric information 331 represented by the tetrahedral mesh can be converted into 3D image information 332 represented by a facet (such as a triangular facet). For example, the tetrahedral mesh can be converted into a plane by a differentiable marching tetrahedron algorithm. Specifically, if two vertices of an edge in the tetrahedral mesh have directed distance values ​​with different signs, a mesh face can be extracted. In this way, the position information in the facet can represent the position of the surface of the target object in the target form in the target space.

[0066] For the texture branch, determine the point in the target space that is located on the target surface. Based on the transformation relationship, similar to formula (1), the reference point corresponding to the point in the reference space can be determined. Based on the texture at the reference point indicated by the reference texture information 323, determine the texture at the point. Exemplarily, for an embodiment in which the 3D image information is represented by a patch, for any point on the patch surface, the reference point corresponding to the point in the reference space can be found using formula (1). The color value of the reference point in the texture field can be determined as the color value of the point and stored as part of the patch.

[0067] The above describes an example process of combining the geometry branch and the texture branch to generate 3D image information. It should be understood that the representation of various information described above is only exemplary and is not intended to limit the scope of the present disclosure.

[0068] In some embodiments, normal information in the reference space can also be constructed to generate finer geometric details, such as hair, clothing wrinkles, etc. Specifically, reference normal information 322 in the reference space can be determined based on the reference geometric information 321, which indicates the normal at each position on the reference surface. Exemplarily, the reference normal information 322 can be represented by a normal field, in which each point on the reference surface has a normal vector. The direction and magnitude of the normal vector represent the orientation and curvature of the surface at that point. For example, the reference normal information can be constructed by calculating the spatial gradient of the signed distance field in the reference space, as shown in the following formula:

[0069]

[0070] Where d(x c ) and n(x c ) are the reference points x in the reference space. c The signed distance value and normal vector of .

[0071] After obtaining the reference normal information 322, the reference point x indicated by the reference normal information can be similarly transformed based on the transformation relationship. c The normal at point x is converted to the target surface d For example, the normal to point x on the target surface d The normal vector n(x d ) can be determined by the following formula:

[0072]

[0073] in Represents vertex v * Relative to the skin weight of the ith joint, R i (θ, β) is B in formula (1) i The rotation component of (θ, β).

[0074] References Figure 3 An example generation process of 3D image information is described, whereby a representation of a target object in any desired target form can be obtained.

[0075] Example Rendering

[0076] As mentioned above, the obtained 3D image information can be rendered into a 3D image under a target perspective. For example, the desired target perspective can be determined based on the camera parameter information. A target image of a target object having a target shape under a target perspective can be rendered based on the 3D image information 332. Taking the 3D image information 332 represented by a facet as an example, refer to Figure 4 Describes an example of rendering.

[0077] Generally speaking, if an implicit neural representation is applied to 3D image information, a neural volume rendering method is used to generate 3D images. However, such neural volume rendering consumes a lot of computing resources, making it difficult to generate high-resolution images. Using patch representation can support efficient rasterization-based rendering, so that high-resolution images can be generated, such as images up to 1024*1024.

[0078] Exemplarily, given camera parameters, i.e., a given viewing angle, a rasterizer can be used to project 3D image information 332 (e.g., a patch) into a 2D mask map 401 and a coordinate map (not shown). It should be understood that any suitable rasterizer can be used. Any pixel on the coordinate map can store the 3D coordinates of the corresponding point on the surface of the patch. Then, for each pixel on the coordinate map, the 3D coordinates of the corresponding point can be converted to a reference space to query the color value from the reference texture information in the reference space. Thus, an image 403 at a given viewing angle, such as a grayscale image or an RGB image, can be obtained.

[0079] In some embodiments, if reference normal information in the reference space is generated, a normal map 402 of the target object at a given viewing angle can also be generated. Each pixel in the normal map 402 represents the normal vector at the corresponding point. In such an embodiment, thanks to the differentiable surface modeling and rendering techniques described above, the normal map at a given viewing angle can be rendered directly from the normal field in the reference space. This allows adversarial training to be performed on the normal map during model training, thereby helping to capture high-frequency geometric details such as hair, wrinkles, etc.

[0080] The above describes rendering for a given perspective. It should be understood that in the model reasoning process, the given perspective is the desired target perspective; in the model training process, the given perspective is the sample perspective.

[0081] The above describes an example generation of 3D image information and an example rendering of a 3D image. In order to more clearly understand the embodiments of the present disclosure, an example application scenario is described below. The user device 110 or the system 105 acquires one or more reference images, such as one or more static images, or frames in a reference video. Exemplarily, the reference image can be input by a user or acquired by an image acquisition device. Morphological recognition is performed on the reference object in the acquired one or more reference images, and the morphology of the recognized reference object is used as the target morphology. Then, through the above reference Figure 2 or Figure 3The described example process can generate 3D image information of the target object in the target form. Additionally, the target perspective can be determined based on the camera perspective in one or more reference images, for example, the camera perspective can be used as the target perspective. In this way, the 3D image information can be rendered according to the target perspective to obtain one or more target images of the 3D image with the target form.

[0082] It should be understood that if the reference objects in different reference images have different shapes (for example, people in the video are moving), the shapes of the objects in the rendered different target images may also be different. Similarly, if the camera perspectives in different reference images are different, different target images may be rendered from different perspectives. In this example scenario, using the embodiments of the present disclosure, the shapes and / or camera perspectives of objects in one or more reference images can be conveniently tracked and reproduced.

[0083] Example training and input optimization

[0084] Reference below Figure 5 The training of the generator for 3D image information (e.g., the geometry mapping network 301, the texture mapping network 302, and the backbone network 303) is described. The generator is used to generate sample 3D image information 500 of the sample object in the sample form. It can be understood that the generation process of the sample 3D image information 500 is similar to the 3D image information 212 and 332 described above, so it will not be repeated.

[0085] like Figure 5 As shown, based on the sample 3D image information 500 and the sample viewing angle related to the camera parameters, a mask map 501 and an image 503 of the sample object under the sample viewing angle can be obtained. The determination of the mask map 501 and the image 503 is similar to the determination of the mask map 401 and the image 403 described above, so they are not repeated here. Using the discriminator 511, a discrimination result 521 for the mask map 501 can be determined, which indicates whether the mask map 501 is a real mask map. Using the discriminator 513, a discrimination result 523 for the image 503 can be determined, which indicates whether the image 503 is a real image. At least based on the discrimination results 521 and the discrimination results 523, an adversarial loss 530 can be determined for updating the generator.

[0086] In some embodiments, if reference normal information is generated, a normal map 502 of the sample object under the sample perspective can be obtained based on the sample 3D three-dimensional image information 500. The determination of the normal map 502 is similar to the normal map 402 described above, so it is not repeated here. Then, the discriminator 512 can be used to determine a discrimination result 522 for the normal map 502. The adversarial loss is further determined based on the discrimination results 521, 521 and 523, so as to update the generator.

[0087] In some embodiments, in addition to the adversarial loss, other losses may be additionally considered to update the generator. For example, in order to make the surface normal vector a unit 2 norm, the Eikonal loss may be applied to the surface indicated by the generated 3D image information. As shown below:

[0088]

[0089] where x i and d(x i ) represent points on the surface and their directed distance values ​​respectively.

[0090] For example, you can use cross entropy loss To regularize the directed distance value in the 3D image information, as shown in the following formula:

[0091]

[0092] Among them, H, σ and sign are binary cross entropy function, sigmoid function and sign function respectively. is the set of edges whose directed distance values ​​between two vertices have different signs.

[0093] In this case, the total loss function used for the generator is as follows:

[0094]

[0095] where λ eik and λ ce are the weights of Eikonal loss and cross entropy loss, Indicates such as Figure 5 Adversarial loss 530 is shown.

[0096] In this way, generators such as the geometry mapping network 301, the texture mapping network 302, and the backbone network 303 can be trained.

[0097] The above describes an example training of the generator. As briefly mentioned above, in some embodiments, input features 201, such as geometric components and texture components, are optimizable. Specifically, description information for the target 3D image can be obtained. The description information can specify at least a portion of the features of the desired 3D image, such as specifying morphological features, texture features, or a combination thereof. For example, the description information can indicate the generation of a person squatting in a red shirt. In some embodiments, the description information can be text. Alternatively or additionally, the description information can include one or more images.

[0098] Initial 3D image information can be generated using initial features for the target object. For example, the initial features can be features randomly sampled from a Gaussian distribution, or can be preset features. The process of generating initial 3D image information is similar to the process described above for 3D image information 212 and 332, so it will not be repeated here. Based on the description information and the initial 3D image information, the difference between the 3D image corresponding to the initial 3D image information and the target 3D image can be determined. Based on the difference, the initial features can be optimized. Specifically, the initial features can be updated based on the difference until a preset condition is met, such as the difference is less than a threshold or the number of iterative updates reaches a threshold number of times. In this way, the updated initial features can be determined as input features 201.

[0099] Depending on the specific form of the description information, the difference can be determined in a corresponding manner. For example, in the case where the description information is text, the initial 3D image information can be used for rendering to obtain a corresponding image, such as described in reference image 403. Then, the image-text matching degree between the obtained image and the text as the description information can be calculated, and the initial features can be optimized based on the image-text matching degree. Alternatively or additionally, the similarity between the text features of the text as the description information and the image features of the obtained image can be calculated, and the initial features can be optimized based on the similarity. In the case where the description information is an image, the initial features can be optimized based on the similarity between the image and the image obtained by rendering.

[0100] It should be understood that, in the case where the input feature 201 includes a geometric component and a texture component, the two components can be optimized simultaneously, or one component can be optimized first and then the other component. In this embodiment, the input feature is optimized based on the desired target 3D image. In this way, the accuracy of the generated 3D image can be further improved.

[0101] Example Process

[0102] Figure 6 FIG. 6 is a flowchart of a process 600 for generating a 3D image according to some embodiments of the present disclosure. The process 600 may be implemented in Figure 2 at system 105.

[0103] At block 610 , the system 105 generates reference information in a reference space corresponding to a reference form based on input features for the target object. The reference information indicates a reference surface of the target object in the reference form.

[0104] In some embodiments, the input features are determined as follows: obtaining description information for the target three-dimensional image; generating initial three-dimensional image information based on the initial features for the target object; determining the difference between the three-dimensional image corresponding to the initial three-dimensional image information and the target three-dimensional image based on the description information and the initial three-dimensional image information; based on the difference, updating the initial features until a preset condition is met; and determining the updated initial features as the input features.

[0105] At block 620, the system 105 determines a target morphology of the target object. In some embodiments, determining the target morphology of the target object includes: identifying a morphology of a reference object in a reference image; and determining the target morphology based on the identified morphology.

[0106] At block 630, the system 105 generates three-dimensional image information of the target object in the target form based on the reference information and the transformation relationship between the target form and the reference form. In some embodiments, the three-dimensional image information is represented by a patch.

[0107] In some embodiments, the input features include a first input component for geometric features and a second input component for texture features. In some embodiments, generating reference information includes: generating reference geometric information in a reference space based on the first input component, the reference geometric information indicating a distance between a point in the reference space and a reference surface; and generating reference texture information in the reference space based on the second input component, the reference texture information indicating a texture corresponding to the point in the reference space.

[0108] In some embodiments, generating three-dimensional image information includes: based on a transformation relationship, converting reference geometric information into target geometric information in a target space corresponding to the target form, the target geometric information indicating whether a point in the target space is on a target surface, the target surface being the surface of the target object in the target form; based on the target geometric information, determining the position of the target surface in the target space as part of the three-dimensional image information; and based on reference texture information, position and transformation relationship, determining the texture of the target surface as part of the three-dimensional image information.

[0109] In some embodiments, determining the texture of the target surface includes: determining a first point in the target space located on the target surface based on the position; determining a second point in the reference space corresponding to the first point based on the transformation relationship; and determining the texture at the first point based on the texture at the second point indicated by the reference texture information.

[0110] In some embodiments, process 600 also includes: determining reference normal information in a reference space based on reference geometric information, the reference normal information indicating normals at various positions on the reference surface; and based on a transformation relationship, converting the normal at a second point indicated by the reference normal information into the normal at the first point as part of the three-dimensional image information.

[0111] In some embodiments, generating reference geometric information includes: using a trained generator to generate a set of geometric feature maps corresponding to three planes respectively from a first input component to characterize the geometric features of the target object in a reference form; and for a point in the reference space, determining the distance between the point and the reference surface based on features in a set of geometric feature maps respectively corresponding to the point.

[0112] In some embodiments, generating reference texture information includes: using a trained generator to generate a set of texture feature maps corresponding to three planes respectively from the second input component to characterize the texture features of the target object in a reference form; and for a point in the reference space, determining the texture corresponding to the point based on the features in a set of texture feature maps respectively corresponding to the point.

[0113] In some embodiments, the reference information is determined by a generator based on the input features. In some embodiments, the training step of the generator includes: using the generator to generate sample three-dimensional image information of the sample object in the sample form; based on the sample three-dimensional image information, obtaining a mask map and an image of the sample object in the sample perspective; using a first discriminator to determine a first discrimination result for the mask map; using a second discriminator to determine a second discrimination result for the image; and updating the generator based on at least the first discrimination result and the second discrimination result.

[0114] In some embodiments, the sample three-dimensional image information also includes normal information of each position of the sample object on the surface of the sample in the sample form. In some embodiments, the training of the generator also includes: obtaining a normal map of the sample object under the sample perspective based on the sample three-dimensional image information; using a third discriminator to determine a third discrimination result for the normal map; and further updating the generator based on the third discrimination result.

[0115] Example devices and equipment

[0116] Figure 7 : A schematic structural block diagram of an apparatus 700 for generating a 3D image according to some embodiments of the present disclosure is shown. The apparatus 700 may be implemented as or included in the implementation system 105, the user device 110, or the 3D image generating device 120. Each module / component in the apparatus 700 may be implemented by hardware, software, firmware, or any combination thereof.

[0117] As shown in the figure, the device 700 includes a reference information generation module 710, which is configured to generate reference information in a reference space corresponding to a reference form based on input features for the target object, and the reference information indicates a reference surface of the target object in the reference form. The device 700 also includes a target form determination module 720, which is configured to determine the target form of the target object. The device 700 also includes an image information generation module 730, which is configured to generate three-dimensional image information of the target object in the target form based on the reference information and the transformation relationship between the target form and the reference form.

[0118] In some embodiments, the input features include a first input component for geometric features and a second input component for texture features, and the reference information generation module 710 includes: a geometric reference information generation module, configured to generate reference geometric information in a reference space based on the first input component, the reference geometric information indicating the distance between a point in the reference space and a reference surface; and a reference texture information generation module, configured to generate reference texture information in the reference space based on the second input component, the reference texture information indicating a texture corresponding to the point in the reference space.

[0119] In some embodiments, the image information generation module 730 includes: an information conversion module, configured to convert reference geometric information into target geometric information in a target space corresponding to the target form based on a transformation relationship, the target geometric information indicating whether a point in the target space is on a target surface, the target surface being the surface of the target object in the target form; a position determination module, configured to determine the position of the target surface in the target space based on the target geometric information as part of the three-dimensional image information; and a texture determination module, configured to determine the texture of the target surface based on reference texture information, position and transformation relationship as part of the three-dimensional image information.

[0120] In some embodiments, the texture determination module is further configured to: determine a first point in the target space located on the target surface based on the position; determine a second point in the reference space corresponding to the first point based on the transformation relationship; and determine the texture at the first point based on the texture at the second point indicated by the reference texture information.

[0121] In some embodiments, the device 700 also includes: a reference normal information determination module, configured to determine reference normal information in a reference space based on reference geometric information, the reference normal information indicating normals at various positions on the reference surface; and a normal conversion module, configured to convert the normal at a second point indicated by the reference normal information into the normal at the first point based on a transformation relationship, as part of the three-dimensional image information.

[0122] In some embodiments, the reference geometric information generation module is further configured to: use a trained generator to generate a set of geometric feature maps corresponding to three planes respectively from the first input component to characterize the geometric features of the target object in the reference form; and for a point in the reference space, determine the distance between the point and the reference surface based on the features in a set of geometric feature maps respectively corresponding to the point.

[0123] In some embodiments, the reference texture information generation module is further configured to: use a trained generator to generate a set of texture feature maps corresponding to the three planes respectively from the second input component to characterize the texture features of the target object in the reference form; and for a point in the reference space, determine the texture corresponding to the point based on the features in a set of texture feature maps respectively corresponding to the point.

[0124] In some embodiments, the device 700 also includes the following modules to determine input features: a description information acquisition module, configured to acquire description information for a target three-dimensional image; an initial three-dimensional image information generation module, configured to generate initial three-dimensional image information based on initial features for the target object; a difference determination module, configured to determine the difference between the three-dimensional image corresponding to the initial three-dimensional image information and the target three-dimensional image based on the description information and the initial three-dimensional image information; an initial feature update module, configured to update the initial features based on the differences until a preset condition is met; and an input feature determination module, configured to determine the updated initial features as input features.

[0125] In some embodiments, the reference information is determined based on the input features using a generator, and the device 700 also includes a sample three-dimensional image information generation module, which is configured to use the generator to generate sample three-dimensional image information of the sample object in the sample form; a mask map and image acquisition module, which is configured to obtain a mask map and an image of the sample object under the sample perspective based on the sample three-dimensional image information; a first result determination module, which is configured to use a first discriminator to determine a first discrimination result for the mask map; a second result determination module, which is configured to use a second discriminator to determine a second discrimination result for the image; and a generator update module, which is configured to update the generator based on at least the first discrimination result and the second discrimination result.

[0126] In some embodiments, the sample three-dimensional image information also includes normal information of each position on the surface of the sample object under the sample form, and the device 700 also includes: a normal map acquisition module, configured to obtain a normal map of the sample object under the sample perspective based on the sample three-dimensional image information; a third result determination module, configured to use a third discriminator to determine a third discrimination result for the normal map; and a generator update module configured to further update the generator based on the third discrimination result.

[0127] In some embodiments, the target morphology determination module 720 is further configured to: identify the morphology of the reference object in the reference image; and determine the target morphology based on the identified morphology.

[0128] In some embodiments, the device 700 also includes: a target perspective determination module, configured to determine the target perspective based on camera parameter information; and a target image rendering module, configured to render a target image of a target object with a target form under a target perspective based on three-dimensional image information.

[0129] In some embodiments, the three-dimensional image information is represented by patches.

[0130] Figure 8 A block diagram of an electronic device 800 is shown in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 8 The electronic device 800 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 8 The electronic device 800 shown can be used to implement Figure 1 system 105, user device 110 or 3D image generating device 120.

[0131] like Figure 8 As shown, the electronic device 800 is in the form of a general electronic device. The components of the electronic device 800 may include, but are not limited to, one or more processors or processing units 810, a memory 820, a storage device 830, one or more communication units 840, one or more input devices 850, and one or more output devices 860. The processing unit 810 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 820. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 800.

[0132] The electronic device 800 typically includes a plurality of computer storage media. Such media can be any accessible media that can be obtained by the electronic device 800, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 820 can be a volatile memory (e.g., a register, a cache, a random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 830 can be a removable or non-removable medium, and can include a machine-readable medium, such as a flash drive, a disk, or any other medium, which can be used to store information and / or data and can be accessed within the electronic device 800.

[0133] The electronic device 800 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 8 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 820 may include a computer program product 825 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0134] The communication unit 840 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 800 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 800 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0135] The input device 850 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 860 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 800 may also communicate with one or more external devices (not shown) through the communication unit 840 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 800, or communicate with any device that allows the electronic device 800 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0136] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0137] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0138] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0139] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0140] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0141] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A method for generating a three-dimensional image, comprising: generating reference information in a reference space corresponding to a reference form based on input features for the target object, the reference information indicating a reference surface of the target object in the reference form; determining a target morphology of the target object; as well as Based on the reference information and the transformation relationship between the target form and the reference form, three-dimensional image information of the target object in the target form is generated.

2. The method according to claim 1, wherein the input feature comprises a first input component for a geometric feature and a second input component for a texture feature, and generating the reference information comprises: generating reference geometric information in the reference space based on the first input component, the reference geometric information indicating a distance between a point in the reference space and the reference surface; as well as Based on the second input component, reference texture information in the reference space is generated, the reference texture information indicating a texture corresponding to a point in the reference space.

3. The method according to claim 2, wherein generating the three-dimensional image information comprises: Based on the transformation relationship, converting the reference geometric information into target geometric information in a target space corresponding to the target form, the target geometric information indicating whether a point in the target space is on a target surface, the target surface being a surface of the target object in the target form; Based on the target geometric information, determine the position of the target surface in the target space as a part of the three-dimensional image information; as well as Based on the reference texture information, the position and the transformation relationship, the texture of the target surface is determined as a part of the three-dimensional image information.

4. The method of claim 3, wherein determining the texture of the target surface comprises: Based on the position, determining a first point in the target space located on the target surface; Based on the transformation relationship, determining a second point in the reference space corresponding to the first point; as well as The texture at the first point is determined based on the texture at the second point indicated by the reference texture information.

5. The method according to claim 4, further comprising: Based on the reference geometric information, determine reference normal information in the reference space, the reference normal information indicating a normal at each position on the reference surface; as well as Based on the transformation relationship, the normal at the second point indicated by the reference normal information is converted into the normal at the first point as a part of the three-dimensional image information.

6. The method according to claim 2, wherein generating the reference geometric information comprises: Using the trained generator, a set of geometric feature maps corresponding to three planes are generated from the first input component to characterize the geometric features of the target object in the reference form; as well as For a point in the reference space, based on the features in the set of geometric feature maps respectively corresponding to the point, the distance between the point and the reference surface is determined.

7. The method according to claim 6, wherein generating the reference texture information comprises: Using the trained generator, a set of texture feature maps corresponding to the three planes are generated from the second input component to characterize the texture features of the target object in the reference form; as well as For a point in the reference space, a texture corresponding to the point is determined based on features in the set of texture feature maps that respectively correspond to the point.

8. The method according to claim 1, wherein the input feature is determined by: Obtaining description information for the target three-dimensional image; Based on the initial features of the target object, generating initial three-dimensional image information; Based on the description information and the initial three-dimensional image information, determining a difference between the three-dimensional image corresponding to the initial three-dimensional image information and the target three-dimensional image; Based on the difference, updating the initial feature until a preset condition is satisfied; and The updated initial features are determined as the input features.

9. The method according to claim 1, wherein the reference information is determined using a generator based on the input features, and the training step of the generator comprises: Using the generator, generating sample three-dimensional image information of the sample object in the sample form; Based on the sample three-dimensional image information, a mask map and an image of the sample object under the sample viewing angle are obtained; Determine a first discrimination result for the mask image using a first discriminator; Determine a second discrimination result for the image using a second discriminator; as well as The generator is updated based on at least the first discrimination result and the second discrimination result.

10. The method according to claim 9, wherein the sample 3D image information further includes normal information of each position of the sample object on the surface of the sample form, and the training of the generator further includes: Based on the three-dimensional image information of the sample, a normal map of the sample object under the sample viewing angle is obtained; Determine a third discrimination result for the normal map by using a third discriminator; as well as The generator is further updated based on the third discrimination result.

11. The method according to claim 1, wherein determining the target form of the target object comprises: identifying a morphology of a reference object in a reference image; as well as Based on the identified morphology, the target morphology is determined.

12. The method according to claim 1, further comprising: Determine the target viewing angle based on the camera parameter information; as well as Based on the three-dimensional image information, a target image of the target object having the target shape at the target viewing angle is rendered.

13. The method according to claim 1, wherein the three-dimensional image information is represented by a patch.

14. A device for generating a three-dimensional image, comprising: A reference information generation module, configured to generate reference information in a reference space corresponding to a reference form based on input features of the target object, wherein the reference information indicates a reference surface of the target object in the reference form; A target shape determination module is configured to determine the target shape of the target object; as well as The image information generating module is configured to generate three-dimensional image information of the target object in the target form based on the reference information and the transformation relationship between the target form and the reference form.

15. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 13 when executed by the at least one processing unit.

16. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 13.

Images