Three-dimensional image generation method and device, electronic equipment and storage medium

Abstract

The invention relates to a three-dimensional image generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a target image which is a two-dimensional image at a target visual angle with a target pixel specification; inputting the target image into a controllable generation model, and triggering the controllable generation model to infer a target basic color map, a target normal map and a target height map; performing map rendering based on the target basic color map, the target normal map and the target height map to obtain a target map; the controllable generation model is an image generation model obtained by training based on a first sample image, a sample basic color map, a sample normal map and a sample height map, and the first sample image has a target pixel specification and corresponds to a target view angle. According to the method and the device, the whole-process automation of the generation process from the standard two-dimensional target image with the target pixel specification and the target view angle to the three-dimensional target map is realized.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and in particular to a method, apparatus, electronic device and storage medium for generating three-dimensional images. Background Technology

[0002] 3D art assets are a crucial type of computer asset, playing a vital role in game development and film production. These assets include character models, scene designs, props, and special effects elements, which, through meticulous modeling and texturing, can present highly realistic visual effects. However, the creation of 3D art assets generally faces problems such as high labor costs, low efficiency, lack of standardization, and inconsistent quality. This is because the generation process of 3D art assets relies heavily on manual labor.

[0003] Specifically, texture assets used for rendering 3D art assets, such as sets of color maps, normal maps, and height maps, currently need to be manually generated by artists, which leads to the following problems: 1. High labor costs: Art staff need to spend a lot of time on image processing, adjustment and repair, which makes it difficult to meet the needs of fast and efficient production.

[0004] 2. Low efficiency and repetitive work: The process of image adjustment, normal map and height map generation involves a lot of repetitive work and manual operation, which can easily lead to fatigue and errors, resulting in low production efficiency.

[0005] 3. Human intervention affects consistency: Differences in the creative styles and processing methods of artists make it difficult to guarantee the quality and consistency of the final generated images. Summary of the Invention

[0006] This disclosure provides a method, apparatus, electronic device, and storage medium for generating three-dimensional images to solve problems in related technologies. The technical solution of this disclosure is as follows: According to a first aspect of the present disclosure, a method for generating a three-dimensional image is provided, the method comprising: Acquire a target image, wherein the target image is a two-dimensional image from a target viewpoint with target pixel specifications; The target image is input into the controllable generation model, which then triggers the model to infer the target base color map, target normal map, and target height map based on the target image. Based on the target base color map, the target normal map and the target height map, texture rendering is performed to obtain a target texture, which is a three-dimensional image from the target's perspective. The controllable generation model is an image generation model trained based on a first sample image, a sample base color map, a sample normal map, and a sample height map. The first sample image has the target pixel specifications and corresponds to the target viewpoint. The sample base color map is used to describe the base color information of the first sample image. The sample normal map is used to describe the normal information of the first sample image, and the normal information is used to represent the surface orientation of each pixel in the first sample image. The sample height map is used to describe the height information of the first sample image, and the height information is used to represent the relative height of each pixel in the first sample image.

[0007] In one exemplary embodiment, prior to acquiring the target image, the method further includes: Obtain an initial image, wherein the initial image is a two-dimensional image with arbitrary pixel specifications and an arbitrary viewpoint; The initial image is input into an image editing model for image editing to obtain the target image. The image editing includes pixel specification normalization and viewpoint adjustment. The image editing model is an image generation model trained based on an image editing instruction used to trigger image editing, a second sample image, and a third sample image. The second sample image has arbitrary pixel specifications and corresponds to arbitrary viewpoints. The third sample image indicates the editing target corresponding to the second sample image, which has the target pixel specifications and the target viewpoint.

[0008] In one exemplary embodiment, the controllable generative model is trained using the following method: Generate color sample pairs, normal sample pairs, and height sample pairs. The color sample pairs include the first sample image and the sample base color map. The normal sample pairs include the first sample image and the sample normal map. The height sample pairs include the first sample image and the sample height map. The controllable generation model is trained based on the color sample pairs and the corresponding color control signals; the controllable generation model is trained based on the normal sample pairs and the corresponding normal control signals; the controllable generation model is trained based on the height sample pairs and the corresponding height control signals.

[0009] In one exemplary embodiment, the color control signal, the normal control signal, and the height control signal are non-text signals.

[0010] In one exemplary embodiment, before training the controllable generative model, the method further includes: The height information corresponding to each pixel in the first sample image is obtained and the height information is normalized to obtain an initial height map; The height information below a preset height threshold in the initial height map is cleared to obtain the sample height map; The first sample image, the sample base color map, the sample normal map, and the sample height map are processed to standardize their pixel specifications.

[0011] In one exemplary embodiment, the image editing instructions include specification adjustment instructions and perspective adjustment instructions; the image editing model includes a feature extraction network and an image generation network, and the image editing model is trained using the following method: The second sample image is input into the image extraction network to obtain the corresponding sample features; The sample features and the second sample image are fused to obtain fused data; The fused data, the specification adjustment command, and the viewpoint adjustment command are input into the image generation network to obtain the image generation result; Based on the difference between the image generation result and the third sample image, the parameters of the feature extraction network and the image generation network are adjusted.

[0012] In one exemplary embodiment, the method further includes, prior to training the image editing model: The resolution of the second sample image and the third sample image is made uniform. The third sample image is cropped to the target pixel size.

[0013] According to a second aspect of the present disclosure, a three-dimensional image generation apparatus is provided, the apparatus comprising: The target image acquisition module is configured to acquire a target image, wherein the target image is a two-dimensional image with target pixel specifications from a target viewpoint; The image generation module is configured to input the target image into a controllable generation model, triggering the controllable generation model to infer the target base color map, target normal map, and target height map based on the target image, respectively. The rendering module is configured to perform texture rendering based on the target base color map, the target normal map and the target height map to obtain a target texture, wherein the target texture is a three-dimensional image from the target's perspective. The controllable generation model is an image generation model trained based on a first sample image, a sample base color map, a sample normal map, and a sample height map. The first sample image has the target pixel specifications and corresponds to the target viewpoint. The sample base color map is used to describe the base color information of the first sample image. The sample normal map is used to describe the normal information of the first sample image, and the normal information is used to represent the surface orientation of each pixel in the first sample image. The sample height map is used to describe the height information of the first sample image, and the height information is used to represent the relative height of each pixel in the first sample image.

[0014] In one exemplary implementation, the target image acquisition module is configured to perform: Obtain an initial image, wherein the initial image is a two-dimensional image with arbitrary pixel specifications and an arbitrary viewpoint; The initial image is input into an image editing model for image editing to obtain the target image. The image editing includes pixel specification normalization and viewpoint adjustment. The image editing model is an image generation model trained based on an image editing instruction used to trigger image editing, a second sample image, and a third sample image. The second sample image has arbitrary pixel specifications and corresponds to arbitrary viewpoints. The third sample image indicates the editing target corresponding to the second sample image, which has the target pixel specifications and the target viewpoint.

[0015] In one exemplary implementation, the image generation module is configured to perform: Generate color sample pairs, normal sample pairs, and height sample pairs. The color sample pairs include the first sample image and the sample base color map. The normal sample pairs include the first sample image and the sample normal map. The height sample pairs include the first sample image and the sample height map. The controllable generation model is trained based on the color sample pairs and the corresponding color control signals; the controllable generation model is trained based on the normal sample pairs and the corresponding normal control signals; the controllable generation model is trained based on the height sample pairs and the corresponding height control signals.

[0016] In one exemplary embodiment, the color control signal, the normal control signal, and the height control signal are non-text signals.

[0017] In one exemplary implementation, the image generation module is configured to perform: The height information corresponding to each pixel in the first sample image is obtained and the height information is normalized to obtain an initial height map; The height information below a preset height threshold in the initial height map is cleared to obtain the sample height map; The first sample image, the sample base color map, the sample normal map, and the sample height map are processed to standardize their pixel specifications.

[0018] In one exemplary embodiment, the image editing instructions include specification adjustment instructions and perspective adjustment instructions; the image editing model includes a feature extraction network and an image generation network, and the image generation module is configured to execute: The second sample image is input into the image extraction network to obtain the corresponding sample features; The sample features and the second sample image are fused to obtain fused data; The fused data, the specification adjustment command, and the viewpoint adjustment command are input into the image generation network to obtain the image generation result; Based on the difference between the image generation result and the third sample image, the parameters of the feature extraction network and the image generation network are adjusted.

[0019] In one exemplary implementation, the image generation module is configured to perform: The resolution of the second sample image and the third sample image is made uniform. The third sample image is cropped to the target pixel size.

[0020] According to a third aspect of the present disclosure, an electronic device is provided, comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the three-dimensional image generation method as described in any of the above embodiments.

[0021] According to a fourth aspect of the present disclosure, a computer storage medium is provided, wherein when instructions in the computer storage medium are executed by a processor of an electronic device, the electronic device performs the three-dimensional image generation method described in any of the above embodiments.

[0022] According to a fifth aspect of the present disclosure, a computer program product is provided, the computer program product including a computer program that, when executed by a processor, implements the three-dimensional image generation method described in any of the above embodiments.

[0023] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects: The 3D image generation method provided in this disclosure can automatically derive a set of target base color maps, target normal maps, and target height maps adapted to a target image through reasoning using a controllable generation model. Based on the derived target base color maps, target normal maps, and target height maps, texture rendering is directly performed to obtain the 3D art assets (target textures) corresponding to the target image. This method automates the entire process of generating 3D target textures from a standard 2D target image with target pixel specifications and target viewpoint, eliminating reliance on manual operation by artists and achieving at least the following three technical effects: 1. Reduce manpower and time costs: No more time is needed for art staff.

[0024] 2. No manual operation required: The generation process of the target base color map, target normal map and target height map is automatically reasoned in the controllable generation model. No manual operation is required, and low production efficiency will not be caused by fatigue and errors of artists.

[0025] 3. High consistency: Differences in the creative styles and processing methods of artists make it difficult to guarantee the quality and consistency of the final generated images. However, the target base color map, target normal map and target height map of this disclosure are all obtained by reasoning using the same controllable generation model, thus possessing consistency and quality stability. As a result, the quality of the 3D art assets rendered based on the target base color map, target normal map and target height map is also consistent and stable.

[0026] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0027] The accompanying drawings, which are incorporated in and form part of this disclosure, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0028] Figure 1 This is a flowchart illustrating a three-dimensional image generation method according to an exemplary embodiment; Figure 2 This is a schematic diagram illustrating a target image acquisition method according to an exemplary embodiment; Figure 3 This is a link diagram of a three-dimensional image generation method according to an exemplary embodiment; Figure 4 This is a flowchart illustrating a training method for a controllable generative model according to an exemplary embodiment; Figure 5This is a schematic diagram illustrating a controllable generative model training process according to an exemplary embodiment; Figure 6 This is a flowchart illustrating a training method for an image editing model according to an exemplary embodiment; Figure 7 This is a flowchart illustrating an image preprocessing method according to an exemplary embodiment; Figure 8 This is a schematic diagram illustrating the image editing model training process according to an exemplary embodiment; Figure 9 This is a block diagram of a three-dimensional image generation apparatus according to an exemplary embodiment; Figure 10 This is a structural block diagram of a computer device according to an exemplary embodiment. Figure 1 ; Figure 11 This is a structural block diagram of a computer device according to an exemplary embodiment. Figure 2 . Detailed Implementation

[0029] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0030] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0031] Figure 1This is a flowchart illustrating a three-dimensional image generation method according to an exemplary embodiment. This three-dimensional image generation method can be applied to an electronic device, which can be implemented independently by a server or a terminal, or jointly by a terminal and a server. The terminal can be, but is not limited to, physical devices such as smartphones, tablets, laptops, desktop computers, smart speakers, smart wearable devices, digital assistants, augmented reality devices, and virtual reality devices, and may also include software such as applications running on the physical device. The server can be, but is not limited to, a standalone server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms, etc. (Refer to...) Figure 1 As shown, the method includes the following steps.

[0032] In step S110, a target image is acquired, wherein the target image is a two-dimensional image with target pixel specifications under a target viewpoint.

[0033] This disclosure does not limit the target pixel specification; it is a pixel specification defined to ensure the generation of standardized 3D art assets. The target pixel specification will vary depending on the specific circumstances and the requirements of the 3D art assets. For example, the target pixel specification could be 1024x1024, 2048*2048, or other suitable pixel specifications. Different application scenarios may have specific requirements for pixel specifications. For instance, in game development, high-resolution pixel specifications can provide finer texture details, while in real-time rendering scenarios, lower pixel specifications may prioritize performance optimization. Furthermore, the selection of the target pixel specification must also consider the processing power of the hardware device and the display characteristics of the final output platform to ensure that the generated 3D art assets maintain high-quality performance in different environments. By flexibly adjusting the target pixel specification, an optimal balance between efficiency and quality can be found.

[0034] This disclosure does not limit the target viewpoint; rather, it refers to viewpoint parameters set to ensure the generation of standardized 3D art assets. The target viewpoint may vary depending on actual needs and application scenarios. For example, the target viewpoint can be a front view, side view, top view, or other specific angles. Different viewpoint choices directly affect the presentation and detail of the 3D model. For instance, in architectural design, combining multiple viewpoints can more comprehensively showcase structural features, while in character modeling, a specific viewpoint may be more effective in highlighting the character's facial expressions or movement details.

[0035] By specifying the target pixel specifications and target viewpoint, a controllable generative model can be trained specifically for these specifications and viewpoints. This model generates a complete set of target base color maps, target normal maps, and target height maps for the target image under these specifications and viewpoints, facilitating the rendering of 3D art assets. In other words, this disclosure allows for the training of a controllable generative model tailored to the target pixel specifications and viewpoints, thereby achieving automated rendering of 3D art assets. Changing the target pixel specifications or using different viewpoints requires training a corresponding controllable generative model. Each combination of target pixel specifications and viewpoints uniquely corresponds to a single controllable generative model.

[0036] In step S120, the target image is input into the controllable generation model, which triggers the controllable generation model to infer the target base color map, target normal map, and target height map based on the target image.

[0037] The target base color map describes the basic hue and color distribution of each pixel in the target image, providing color information support for the rendering of the target image. The target normal map, by capturing the normal direction information of each pixel in the target image, reflects the lighting characteristics of the model surface, thereby enhancing the detail of the 3D model. The target height map describes the height information of each pixel in the target image, providing accurate geometric structural references for subsequent 3D modeling. These generated image data work together to significantly improve the rendering effect of 3D art assets and ensure that the final output model is visually more realistic and meets the expected design requirements.

[0038] Current technologies require manual creation of a complete set of target base color maps, target normal maps, and target height maps based on the target image. This leads to high labor and time costs, low efficiency, and unstable image quality. Even when some technologies propose automatic image generation solutions, they can only generate single base color maps, normal maps, or height maps. Images that cannot be automatically generated still rely on manual labor, and it is difficult for manual labor to generate images with a consistent style as machine-generated images. This results in difficulties in manual processing, poor image consistency, and unstable quality of 3D art assets. In short, the inability of current technologies to generate a complete set of base color maps, normal maps, and height maps prevents the resolution of the consistency and instability issues in the image quality of 3D art assets.

[0039] This disclosure trains a controllable generation model corresponding to both target pixel specifications and target viewpoint. This single model can automatically perform three inferences to obtain a complete set of target base color maps, target normal maps, and target height maps, thereby significantly reducing the need for manual intervention and greatly improving generation efficiency. Through the automated inference process, not only is the workload of artists reduced, but the consistency and high quality of the output results are also ensured. Furthermore, this method can process a large number of target images in a short time, adapting to the needs of different application scenarios, thus further optimizing the overall workflow of 3D image generation.

[0040] In order to automatically output a complete set of target base color map, target normal map, and target height map for a target image through three inferences, the controllable generation model in this disclosure is an image generation model trained based on a first sample image, a sample base color map, a sample normal map, and a sample height map. The first sample image has the target pixel specifications and corresponds to the target viewpoint. The sample base color map is used to describe the base color information of the first sample image. The sample normal map is used to describe the normal information of the first sample image, and the normal information is used to represent the surface orientation of each pixel in the first sample image. The sample height map is used to describe the height information of the first sample image, and the height information is used to represent the relative height of each pixel in the first sample image.

[0041] This disclosure does not limit the technical paradigm corresponding to the image generation model. For example, it can belong to one of the following: Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), or Diffusion Models. Among them, Diffusion Models are a technology that has achieved significant breakthroughs in recent years, especially in generating high-quality, detailed images, where they have demonstrated powerful capabilities. Its core advantages lie in high-quality image generation, high stability, flexible editing, and controllability, enabling it to quickly and efficiently empower various application scenarios. Typical downstream applications include image editing and controllable image generation. Image editing capabilities allow the user to edit the original image and generate an edited image based on a given input image and a reference text command. Controllable image generation allows the user to customize the controller based on the input original image to obtain the desired controlled image.

[0042] In one exemplary implementation, the image generation model used to train the controllable generative model can be a diffusion model that accepts semantic control. This diffusion model can precisely control the generation of an image based on the input semantic information, generating an image that meets expectations. By introducing a semantic control mechanism, the input image can be reasoned separately in terms of color, normal, and height dimensions, generating three types of images: a base color map, a normal map, and a height map.

[0043] In step S130, texture rendering is performed based on the target base color map, the target normal map, and the target height map to obtain a target texture, which is a three-dimensional image from the target's perspective.

[0044] Texture rendering can be performed using the target base color map, the target normal map, and the target height map to obtain a 3D art asset—the target texture. This disclosure will not elaborate on this process. During texture rendering, the target base color map, target normal map, and target height map are first loaded into the rendering engine. The rendering engine parses these images based on their semantic information and maps them to the corresponding positions on the 3D model. The target base color map is mainly used to define the color and texture details of the model's surface, ensuring that the visual effect meets the expected design. The target normal map enhances the model's lighting and shadow performance by recording information about the surface normal direction, making it look more three-dimensional and realistic. The target height map simulates the uneven details of the model's surface, generating a more refined geometric structure through displacement mapping technology. Finally, these images undergo a series of complex calculations and synthesis in the rendering engine to generate a 3D image from the target's perspective, i.e., the target texture, thereby significantly reducing the workload of manual adjustments while improving generation efficiency and quality.

[0045] The 3D image generation method provided in this disclosure can automatically derive a set of target base color maps, target normal maps, and target height maps adapted to a target image through reasoning using a controllable generation model. Based on the derived target base color maps, target normal maps, and target height maps, texture rendering is directly performed to obtain the 3D art assets (target textures) corresponding to the target image. This method automates the entire process of generating 3D target textures from a standard 2D target image with target pixel specifications and target viewpoint, eliminating reliance on manual operation by artists and achieving at least the following three technical effects: 1. Reduce manpower and time costs: No more time is needed for art staff.

[0046] 2. No manual operation required: The generation process of the target base color map, target normal map and target height map is automatically reasoned in the controllable generation model. No manual operation is required, and low production efficiency will not be caused by fatigue and errors of artists.

[0047] 3. High consistency: Differences in the creative styles and processing methods of artists make it difficult to guarantee the quality and consistency of the final generated images. However, the target base color map, target normal map and target height map of this disclosure are all obtained by reasoning using the same controllable generation model, thus possessing consistency and quality stability. As a result, the quality of the 3D art assets rendered based on the target base color map, target normal map and target height map is also consistent and stable.

[0048] In one exemplary implementation, please refer to Figure 2 The diagram illustrates a target image acquisition method according to this disclosure. Prior to acquiring the target image, the method further includes: S210. Obtain an initial image, wherein the initial image is a two-dimensional image with arbitrary pixel specifications and an arbitrary viewing angle.

[0049] S220. The initial image is input into an image editing model for image editing to obtain the target image. The image editing includes pixel specification normalization and viewpoint adjustment. The image editing model is an image generation model trained based on an image editing instruction for triggering image editing, a second sample image, and a third sample image. The second sample image has arbitrary pixel specifications and corresponds to arbitrary viewpoints. The third sample image indicates the editing target corresponding to the second sample image, which has the target pixel specifications and the target viewpoint.

[0050] This disclosure does not limit the technical paradigm corresponding to the image editing model. For example, it may belong to one of the following: Generative Adversarial Network (GAN), Variational Autoencoder (VAE), Diffusion Model, etc.

[0051] This image editing model can transform input images of arbitrary pixel specifications and viewpoints into output images that meet the target pixel specifications and viewpoints, ensuring that the output of the image editing model is standardized in both pixel specifications and viewpoints. By specifying pixel specification normalization and viewpoint adjustment as two image editing actions, an image editing model can be trained to perform image editing using these two actions. This image editing model is specifically designed to execute these two editing actions on any image of arbitrary pixel specifications and viewpoints, and outputs an image with the target pixel specifications and viewpoints. In other words, this disclosure allows for the training of a targeted image editing model based on the target pixel specifications and viewpoints. Changing the target pixel specifications or different target viewpoints requires training a corresponding image editing model. Each combination of target pixel specifications and target viewpoints can uniquely correspond to an image editing model.

[0052] This process not only improves image processing efficiency but also ensures that any image can generate a standardized image with corresponding pixel specifications and viewpoints, which can then be used to generate a set of corresponding base color maps, normal maps, and height maps. Through the image editing model and the previously mentioned controllable generation model, the entire process from input images of arbitrary pixel specifications and viewpoints to the final generation of 3D art assets can be fully automated. Both the image editing model and the controllable generation model have a unique correspondence with the combination of target pixel specifications and target viewpoints, ensuring that the entire 3D image generation process is subject to the standardized constraints of the target pixel specifications and target viewpoints. This standardized constraint effectively reduces the uncertainty caused by human intervention, making the generated 3D art assets more stable and reliable in quality. Furthermore, the close integration of the image editing model and the controllable generation model avoids the error accumulation problem caused by multiple manual adjustments in traditional methods, thereby improving the consistency of the final output results.

[0053] In one exemplary implementation, please refer to Figure 3 This diagram illustrates the link diagram of the three-dimensional image generation method in this disclosure. The link includes the following core components: First, you can input any irregular image with any pixel size and any viewpoint that is larger than the target pixel size, without any manual processing.

[0054] Then, the irregular image is input into the image editing model. The image editing model will perform image editing on the irregular image, using image editing instructions including pixel specification normalization instructions and perspective adjustment instructions.

[0055] Specifically, the pixel specification normalization instruction can be "crop the image to a standard 1024*1024 image." By executing this instruction, the image editing model crops the irregular image to obtain an intermediate image. The image editing model then executes a perspective adjustment instruction on this intermediate image: "adjust the image's perspective from any angle to a neutral angle," thereby generating a 1024*1024 image with a neutral perspective. Here, 1024*1024 represents the target pixel specification, and the neutral perspective represents the target viewing angle.

[0056] Next, the 1024*1024 front-view image is input into the controllable generation model. The controllable generation model infers three times in succession according to the "color map" mode, "normal map" mode, and "height map" mode, respectively, to obtain a set of three basic color maps, normal maps, and height maps; the size of these three maps is 1024*1024.

[0057] Finally, these three images are fed into the rendering engine to render the required 3D art assets.

[0058] Clearly, the execution chain of this 3D image generation method achieves end-to-end automation, from generating any image with pixel specifications larger than the target pixel specifications and any viewpoint, to the generation of 3D art assets. This chain uses the target pixel specifications and target viewpoint as constraints to ensure the standardization and normalization of the image editing process, as well as the generation of basic color maps, normal maps, and height maps, greatly improving image generation efficiency and quality. This fully automated process not only reduces errors that may be caused by human intervention but also significantly shortens the production cycle. Furthermore, due to the collaborative work of the image editing model and the controllable generation model, the entire process ensures the consistency and high precision of the output results, meeting the needs of different fields for 3D art assets.

[0059] This disclosure does not limit the training method of the controllable generative model. For example, mainstream techniques in deep learning, such as generative adversarial networks or variational autoencoders, can be used for training. These methods can learn the latent feature distribution of images through a large number of data samples, thereby generating a suitable base color map, normal map, and height map based on the input conditions during the inference stage. Please refer to [reference needed]. Figure 4 The diagram illustrates a flowchart of a training method for a controllable generative model in an exemplary embodiment of this disclosure. The controllable generative model is trained using the following method: S410. Generate color sample pairs, normal sample pairs, and height sample pairs, wherein the color sample pairs include the first sample image and the sample base color map, the normal sample pairs include the first sample image and the sample normal map, and the height sample pairs include the first sample image and the sample height map.

[0060] The first sample image in this disclosure has target pixel specifications and target viewpoint. This disclosure does not limit the number of first sample images. The sample base color map, sample normal map, and sample height map corresponding to each first sample image can be manually created or can come from image data corresponding to existing 3D art assets.

[0061] To improve training effectiveness and optimize the quality of the sample base color map, sample normal map, and sample height map, the height information corresponding to each pixel in the first sample image can be obtained and normalized to obtain an initial height map; height information in the initial height map that is less than a preset height threshold is removed to obtain the sample height map; and the first sample image, the sample base color map, the sample normal map, and the sample height map are subjected to pixel specification unification processing.

[0062] Of course, this disclosure does not limit the height threshold, and the specific value of the height threshold can be adjusted according to actual needs. For example, when processing detailed 3D models, a lower height threshold can be selected to retain more detail information; while when processing coarser models, the height threshold can be appropriately increased to reduce unnecessary noise interference.

[0063] S420. Train the controllable generation model based on the color sample pairs and the corresponding color control signals; train the controllable generation model based on the normal sample pairs and the corresponding normal control signals; train the controllable generation model based on the height sample pairs and the corresponding height control signals.

[0064] For different first sample images, the color control signals of their associated color sample pairs are the same, and similarly, the normal control signals of their associated normal sample pairs and the height control signals of their associated height sample pairs are also consistent. However, for the same first sample image, the color control signals of its associated color sample pairs, the normal control signals of its associated normal sample pairs, and the height control signals of its associated height sample pairs are all different signals.

[0065] In one exemplary embodiment, this disclosure proposes that textual semantics are insufficient to accurately describe the tasks of generating a base color map, a normal map, and a height map. Therefore, the color control signal, the normal control signal, and the height control signal are non-textual signals; for example, they can be represented using specific codes without explicit physical meaning. This design aims to avoid the semantic influence of the textual signals themselves, reduce interference caused by the difference between the semantics of the textual signals and the true meaning of their corresponding reasoning tasks, thereby improving the training effect and accuracy of the controllable generative model and accelerating the convergence speed. By using non-textual signals, the model can focus more on feature learning of the image data itself, rather than being potentially misled by textual semantics.

[0066] This disclosure does not limit the relationship between the three training steps: training the controllable generation model based on the color sample pairs and corresponding color control signals; training the controllable generation model based on the normal sample pairs and corresponding normal control signals; and training the controllable generation model based on the height sample pairs and corresponding height control signals. These steps can be implemented sequentially, in parallel, or in combination, and this disclosure does not limit them.

[0067] For sequential implementation, corresponding to the same first sample image, parameter tuning is first performed based on color sample pairs and their corresponding color control signals, followed by parameter tuning based on normal sample pairs and their normal control signals, and finally parameter tuning based on height sample pairs and their height control signals. This sequential approach helps to progressively build the model's capabilities, from basic color generation to more complex normal and height generation, with each stage providing a solid foundation for the next. For parallel implementation, color sample pairs, normal sample pairs, and height sample pairs can be input into different training modules, but these modules share the same feature extraction layer. During parallel implementation, each training module independently processes its corresponding sample pairs and control signals, while simultaneously achieving information exchange and collaborative optimization through the shared feature extraction layer. In other words, each training module and its corresponding feature extraction layer perform targeted parameter tuning based on their respective training results. This approach not only accelerates training speed but also ensures consistency in feature representation across different tasks. The integrated implementation is similar to the parallel implementation, except that the losses generated by the training results of each training module are accumulated, and the parameters of all training modules are adjusted based on the accumulated total loss. This approach can better capture the correlation between different features, thereby improving the model's overall generation capability.

[0068] This disclosure does not limit the loss function used in the training process or the training stopping condition. For example, common loss functions such as mean squared error or cross-entropy can be used, and the training stopping condition can be set according to actual needs, such as reaching a predetermined number of iterations or the model performance meeting a specific threshold.

[0069] This disclosure ensures, through the design of its training method, that the model can effectively learn the ability to make consistent inferences about the base color map, normal map, and height map for the same image during training. This allows the trained model to automatically generate the expected target base color map, target normal map, and target height map, while also possessing strong generalization ability and being suitable for various application scenarios. Furthermore, this method significantly reduces the need for manual intervention, further optimizing the overall workflow.

[0070] Please refer to the following. Figure 5 This illustration shows a schematic diagram of a controllable generative model training process according to a specific embodiment of this disclosure. This disclosure details a controllable generative model training process under a specific example: S510. Prepare the training data source, which includes more than 6,000 samples. Each sample includes the original image, color map, normal map, and height map with corresponding relationships. S520. Prepare training data and clean the data. Clean and process the data, normalize the height information of the height map to the 0-1 range, and remove the data that are distributed below 0.1. This is because these removed data have almost no height information, which affects the training fit. S530. Perform pixel standardization on the four paired images in each sample to maintain a resolution of 1024*1024; pair the original image with the normal map, height map, and color map respectively, that is, change from a 1-to-3 single sample to 3 sets of 1-to-1 sample pairs. S540. Using three sets of one-to-one sample pairs generated for each sample, perform multiple rounds of model training based on a text-controlled diffusion model until training is complete.

[0071] This disclosure does not limit the training method of the image editing model. For example, mainstream techniques in deep learning such as generative adversarial networks or variational autoencoders can be used for training. These methods can learn the latent feature distribution of images through a large number of data samples, thereby enabling pixel-level editing and viewpoint adjustment of the input image during the inference stage. Please refer to [reference needed]. Figure 6 The diagram illustrates a flowchart of a training method for an image editing model in an exemplary embodiment of this disclosure. The image editing instructions include specification adjustment instructions and viewpoint adjustment instructions; the image editing model includes a feature extraction network and an image generation network, and the image editing model is trained using the following method: S610. Input the second sample image into the image extraction network to obtain the corresponding sample features.

[0072] This disclosure does not limit the image extraction network; convolutional neural networks or other deep learning models can be used to achieve efficient feature extraction. Through the combination and optimization of multi-layer neural networks, the image extraction network can automatically identify and capture key information in the input image, providing high-quality feature representations for subsequent image generation. For example, this disclosure can use a 12-layer VIT as the image extraction network. VIT refers to a visual model based on the Transformer architecture, which can effectively capture global feature information in an image. By segmenting the image into fixed-size blocks and linearly embedding them into a high-dimensional space, this network structure can process data in parallel across multiple attention heads, thereby improving the efficiency and accuracy of feature extraction. Furthermore, VIT utilizes a large-scale dataset for pre-training during training, giving it strong generalization ability when faced with diverse inputs, making it particularly suitable for complex 3D image generation tasks.

[0073] In one exemplary implementation, please refer to [the relevant documentation] before training the image editing model. Figure 7The diagram illustrates a flowchart of the image preprocessing method of this disclosure, the method further comprising: S710. Perform resolution homogenization processing on the second sample image and the third sample image; S720. Crops the third sample image to the target pixel size.

[0074] The arbitrary pixel specification mentioned in this disclosure refers to any pixel specification that is higher than or equal to the target pixel specification. The second sample image has arbitrary pixel specifications and arbitrary viewpoints. The purpose of this step is to ensure the consistency and standardization of the second and third sample images, thereby improving the training effect and generation quality of the model. By ensuring that the second and third sample images have the same resolution and cropping the third sample image to the target pixel specification, the trained image editing model can have standardized image editing capabilities for the target pixel specifications and target viewpoints.

[0075] S620. The sample features and the second sample image are fused to obtain fused data.

[0076] This disclosure does not limit the fusion method; for example, weighted averaging, stitching, or more complex attention mechanisms can be used to fuse features with the image. Based on extensive experiments, this disclosure proposes that stitching extracted features with the image itself can significantly improve the quality and detail of the generated image. This method not only preserves the spatial information of the original image but also further optimizes the generated result through feature enhancement.

[0077] S630. Input the fused data, the specification adjustment command, and the viewpoint adjustment command into the image generation network to obtain the image generation result.

[0078] This disclosure first uses a specification adjustment command to adjust the pixel specifications of the fused data, and then uses a viewpoint adjustment command to adjust the viewpoint of the data after pixel specification adjustment. It is important to note that this disclosure does not perform image editing on a single image or feature, but rather on the fusion result of an image and features, such as a stitched result, to achieve more complex and refined 3D image editing effects, thereby accelerating the convergence speed of the training process and improving the quality of the generated image. This disclosure does not limit the image generation network; it can be a diffusion model or a generative adversarial network (GAN).

[0079] S640. Based on the difference between the image generation result and the third sample image, adjust the parameters of the feature extraction network and the image generation network.

[0080] This disclosure does not limit the method of adjusting the parameters of the feature extraction network and the image generation network based on the difference between the generated image and the third sample image. For example, gradient descent or a variant thereof can be used to optimize the parameters. By calculating the gradient of the loss function, the weights and biases in the network are updated, thereby gradually reducing the difference between the generated image and the third sample image. Furthermore, an adaptive learning rate method, such as the Adam optimizer, can be introduced to improve training efficiency and stability. This disclosure does not limit the loss function and training stopping condition. For example, the loss function can be mean squared error, cross-entropy loss, or other custom forms suitable for image generation tasks. The training stopping condition can be determined based on a preset number of iterations, a convergence threshold for the loss value, or performance metrics on the validation set.

[0081] Please refer to the following. Figure 8 This illustration shows a schematic diagram of the image editing model training process according to a specific embodiment of the present disclosure. The present disclosure details the image editing model training process under a specific example: S810. Prepare the training data source, which includes more than 3200 samples. Each sample includes a regular graph and an irregular graph with corresponding relationships. The regular graph has the target perspective. S820. Prepare training data and clean the data. Perform data cleaning and processing on the above data. Crop the regular and irregular graphs to ensure that both have the same resolution of 1024. Perform center cropping on the long graph. S830. Determine the specification adjustment command and the view adjustment command. The specification adjustment command can be "Please adjust the pixel specification to 1024*1024", and the view adjustment command can be "Adjust the view of the input image from any view to the positive view".

[0082] S840. Based on specification adjustment instructions, perspective adjustment instructions, regular graphs, and irregular graphs, perform multiple rounds of model training on the text-controlled diffusion model until the training is completed.

[0083] The three-dimensional image generation scheme mentioned in this disclosure has the following significant characteristics: 1. Controllable Texture Generation: Common texture generation methods do not support image reference input or arbitrary image reference input. This solution introduces an image input (target image or initial image) as a seed anchor point, enabling more controllable texture generation. It incorporates the image editing capabilities of the diffusion model into the texture generation process, resulting in high quality, high stability, and high controllability. 2. End-to-end texture component generation method: Train a matching controllable generation model and image editing model that can generate a complete set of normal maps, height maps, and base color maps, making the entire generation process fully automatic and highly controllable; while related technologies can at most generate a single normal map, height map, or base color map, and cannot generate a complete set of normal maps, height maps, and base color maps, so they still have significant deficiencies in terms of controllability. Moreover, generating a single normal map, height map, or base color map will still cause the images that cannot be automatically generated to face consistency problems, and will still result in poor quality of rendered 3D art assets.

[0084] 3. Image Consistency Guarantee: By combining diffusion model generation and editing techniques, the complete set of normal maps, height maps, and base color maps generated in this disclosure exhibit high consistency in detail and style, thereby generating high-quality 3D art assets. This technology is particularly suitable for large-scale game development scenarios that require a large number of image resources and a unified style in scene art production. Because it can maintain consistency in style, quality, and detail for each generated image, it avoids the quality instability caused by differences in creative style during manual production.

[0085] Taking the application of this disclosure in texture creation for game development as an example, the technical advantages of this disclosure are analyzed as follows: First, this disclosure can significantly reduce the time artists spend in the texture generation process, especially in the generation of normal maps and height maps. It eliminates a lot of repetitive work and detail adjustment, allowing the art team to focus more on creative design and saving manpower, time and energy costs.

[0086] Furthermore, this disclosure ensures the consistency of the quality of the generated set of images, which not only improves the accuracy of the generated effect but also guarantees the uniformity of the artistic style of the textures.

[0087] Figure 9 This is a block diagram illustrating a three-dimensional image generation apparatus according to an exemplary embodiment. (Refer to...) Figure 9 The device includes: The target image acquisition module 910 is configured to acquire a target image, wherein the target image is a two-dimensional image with target pixel specifications from a target viewpoint. The image generation module 920 is configured to input the target image into a controllable generation model, triggering the controllable generation model to infer a target base color map, a target normal map, and a target height map based on the target image. The rendering module 930 is configured to perform texture rendering based on the target base color map, the target normal map and the target height map to obtain a target texture, wherein the target texture is a three-dimensional image from the target's perspective. The controllable generation model is an image generation model trained based on a first sample image, a sample base color map, a sample normal map, and a sample height map. The first sample image has the target pixel specifications and corresponds to the target viewpoint. The sample base color map is used to describe the base color information of the first sample image. The sample normal map is used to describe the normal information of the first sample image, and the normal information is used to represent the surface orientation of each pixel in the first sample image. The sample height map is used to describe the height information of the first sample image, and the height information is used to represent the relative height of each pixel in the first sample image.

[0088] In one exemplary embodiment, the target image acquisition module 910 is configured to perform: Obtain an initial image, wherein the initial image is a two-dimensional image with arbitrary pixel specifications and an arbitrary viewpoint; The initial image is input into an image editing model for image editing to obtain the target image. The image editing includes pixel specification normalization and viewpoint adjustment. The image editing model is an image generation model trained based on an image editing instruction used to trigger image editing, a second sample image, and a third sample image. The second sample image has arbitrary pixel specifications and corresponds to arbitrary viewpoints. The third sample image indicates the editing target corresponding to the second sample image, which has the target pixel specifications and the target viewpoint.

[0089] In one exemplary embodiment, the image generation module 920 is configured to perform: Generate color sample pairs, normal sample pairs, and height sample pairs. The color sample pairs include the first sample image and the sample base color map. The normal sample pairs include the first sample image and the sample normal map. The height sample pairs include the first sample image and the sample height map. The controllable generation model is trained based on the color sample pairs and the corresponding color control signals; the controllable generation model is trained based on the normal sample pairs and the corresponding normal control signals; the controllable generation model is trained based on the height sample pairs and the corresponding height control signals.

[0090] In one exemplary embodiment, the color control signal, the normal control signal, and the height control signal are non-text signals.

[0091] In one exemplary embodiment, the image generation module 920 is configured to perform: The height information corresponding to each pixel in the first sample image is obtained and the height information is normalized to obtain an initial height map; The height information below a preset height threshold in the initial height map is cleared to obtain the sample height map; The first sample image, the sample base color map, the sample normal map, and the sample height map are processed to standardize their pixel specifications.

[0092] In one exemplary embodiment, the image editing instructions include specification adjustment instructions and perspective adjustment instructions; the image editing model includes a feature extraction network and an image generation network, and the image generation module 920 is configured to execute: The second sample image is input into the image extraction network to obtain the corresponding sample features; The sample features and the second sample image are fused to obtain fused data; The fused data, the specification adjustment command, and the viewpoint adjustment command are input into the image generation network to obtain the image generation result; Based on the difference between the image generation result and the third sample image, the parameters of the feature extraction network and the image generation network are adjusted.

[0093] In one exemplary embodiment, the image generation module 920 is configured to perform: The resolution of the second sample image and the third sample image is made uniform. The third sample image is cropped to the target pixel size.

[0094] Regarding the apparatus in the above embodiments, the specific manner of each step has been described in detail in the embodiments of the foregoing method, and will not be elaborated here.

[0095] Please refer to Figure 10 It illustrates the structural block of a computer device provided in an exemplary embodiment of this disclosure. Figure 1 The computer device may be a terminal. This computer device is used to implement the three-dimensional image generation method provided in the above embodiments. Specifically: Typically, computer device 1000 includes a processor 1001 and a memory 1002.

[0096] Processor 1001 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 1001 may be implemented using at least one hardware form selected from DSP (Digital Signal Processing), FPGA (Field Programmable Gate Array), and PLA (Programmable Logic Array). Processor 1001 may also include a main processor and a coprocessor. The main processor, also known as a CPU (Central Processing Unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In an exemplary embodiment, processor 1001 may integrate a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the screen. In an exemplary embodiment, processor 1001 may also include an AI (Artificial Intelligence) processor, which is used to handle computational operations related to machine learning.

[0097] The memory 1002 may include one or more computer-readable storage media, which may be non-transitory. The memory 1002 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In an exemplary embodiment, the non-transitory computer-readable storage medium in the memory 1002 is used to store at least one instruction, at least one program, code set, or instruction set, configured to be executed by one or more processors to implement the above-described three-dimensional image generation method.

[0098] In one exemplary embodiment, the computer device 1000 may optionally include a peripheral device interface 1003 and at least one peripheral device. The processor 1001, memory 1002, and peripheral device interface 1003 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 1003 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 1004, a touch display screen 1005, a camera assembly 1006, an audio circuit 1007, a positioning assembly 1008, and a power supply 1009.

[0099] Those skilled in the art will understand that Figure 10The structure shown does not constitute a limitation on the computer device 1000, and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0100] Please refer to Figure 11 It illustrates the structural block of a computer device provided in another exemplary embodiment of this disclosure. Figure 2 The computer device can be a server used to execute the aforementioned 3D image generation method. Specifically: Computer device 1100 includes a central processing unit (CPU) 1101, a system memory 1104 including random access memory (RAM) 1102 and read-only memory (ROM) 1103, and a system bus 1105 connecting the system memory 1104 and the CPU 1101. Computer device 1100 also includes a basic input / output system (I / O system) 1106 that facilitates information transfer between various devices within the computer, and a mass storage device 1107 for storing the operating system 1113, application programs 1114, and other program modules 1111.

[0101] The basic input / output system 1106 includes a display 1108 for displaying information and an input device 1109 for user input, such as a mouse or keyboard. Both the display 1108 and the input device 1109 are connected to the central processing unit 1101 via an input / output controller 1110 connected to the system bus 1105. The basic input / output system 1106 may also include the input / output controller 1110 for receiving and processing input from multiple other devices such as a keyboard, mouse, or electronic stylus. Similarly, the input / output controller 1110 also provides output to a display screen, printer, or other types of output devices.

[0102] Mass storage device 1107 is connected to central processing unit 1101 via a mass storage controller (not shown) connected to system bus 1105. Mass storage device 1107 and its associated computer-readable media provide non-volatile storage for computer device 1100. That is, mass storage device 1107 may include computer-readable media (not shown) such as hard disk or CD-ROM (Compact Disc Read-Only Memory) drive.

[0103] Without loss of generality, computer-readable media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes RAM, ROM, EPROM (Erasable Programmable Read Only Memory), EEPROM (Electrically Erasable Programmable Read Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that computer storage media are not limited to the above-mentioned types. The system memory 1104 and mass storage device 1107 described above can be collectively referred to as memory.

[0104] According to various embodiments of this disclosure, computer device 1100 can also be connected to a remote computer on a network, such as the Internet. That is, computer device 1100 can be connected to network 1112 via network interface unit 1111 connected to system bus 1105, or network interface unit 1111 can be used to connect to other types of networks or remote computer systems (not shown).

[0105] The aforementioned memory also includes a computer program stored in the memory and configured to be executed by one or more processors to implement the aforementioned three-dimensional image generation method.

[0106] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is executed by a processor to implement the three-dimensional image generation method.

[0107] Optionally, the computer-readable storage medium may include: ROM (Read Only Memory), RAM (Random Access Memory), SSD (Solid State Drives), or optical disc, etc. The random access memory may include ReRAM (Resistance Random Access Memory) and DRAM (Dynamic Random Access Memory).

[0108] In an exemplary embodiment, a computer-readable storage medium including program code is also provided, such as a memory including program code, which can be executed by a processor to complete the above-described three-dimensional image generation method. Optionally, the computer-readable storage medium may be read-only memory (ROM), random access memory (RAM), compact-disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0109] In an exemplary embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the three-dimensional image generation method described above.

[0110] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0111] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for generating a three-dimensional image, characterized in that, The method includes: Acquire a target image, wherein the target image is a two-dimensional image from a target viewpoint with target pixel specifications; The target image is input into the controllable generation model, which then triggers the model to infer the target base color map, target normal map, and target height map based on the target image. Based on the target base color map, the target normal map and the target height map, texture rendering is performed to obtain a target texture, which is a three-dimensional image from the target's perspective. The controllable generation model is an image generation model trained based on a first sample image, a sample base color map, a sample normal map, and a sample height map. The first sample image has the target pixel specifications and corresponds to the target viewpoint. The sample base color map is used to describe the base color information of the first sample image. The sample normal map is used to describe the normal information of the first sample image, and the normal information is used to represent the surface orientation of each pixel in the first sample image. The sample height map is used to describe the height information of the first sample image, and the height information is used to represent the relative height of each pixel in the first sample image.

2. The method according to claim 1, characterized in that, Before acquiring the target image, the method further includes: Obtain an initial image, wherein the initial image is a two-dimensional image with arbitrary pixel specifications and an arbitrary viewpoint; The initial image is input into an image editing model for image editing to obtain the target image. The image editing includes pixel specification normalization and viewpoint adjustment. The image editing model is an image generation model trained based on an image editing instruction used to trigger image editing, a second sample image, and a third sample image. The second sample image has arbitrary pixel specifications and corresponds to arbitrary viewpoints. The third sample image indicates the editing target corresponding to the second sample image, which has the target pixel specifications and the target viewpoint.

3. The method according to claim 1, characterized in that, The controllable generative model is trained using the following method: Generate color sample pairs, normal sample pairs, and height sample pairs. The color sample pairs include the first sample image and the sample base color map. The normal sample pairs include the first sample image and the sample normal map. The height sample pairs include the first sample image and the sample height map. The controllable generation model is trained based on the color sample pairs and the corresponding color control signals; the controllable generation model is trained based on the normal sample pairs and the corresponding normal control signals; the controllable generation model is trained based on the height sample pairs and the corresponding height control signals.

4. The method according to claim 3, characterized in that, The color control signal, the normal control signal, and the height control signal are non-text signals.

5. The method according to claim 3, characterized in that, Before training the controllable generative model, the method further includes: The height information corresponding to each pixel in the first sample image is obtained and the height information is normalized to obtain an initial height map; The height information below a preset height threshold in the initial height map is cleared to obtain the sample height map; The first sample image, the sample base color map, the sample normal map, and the sample height map are processed to standardize their pixel specifications.

6. The method according to claim 2, characterized in that, The image editing instructions include specification adjustment instructions and perspective adjustment instructions; the image editing model includes a feature extraction network and an image generation network, and the image editing model is trained using the following method: The second sample image is input into the image extraction network to obtain the corresponding sample features; The sample features and the second sample image are fused to obtain fused data; The fused data, the specification adjustment command, and the viewpoint adjustment command are input into the image generation network to obtain the image generation result; Based on the difference between the image generation result and the third sample image, the parameters of the feature extraction network and the image generation network are adjusted.

7. The method according to claim 6, characterized in that, Before training the image editing model, the method further includes: The resolution of the second sample image and the third sample image is made uniform. The third sample image is cropped to the target pixel size.

8. A three-dimensional image generation device, characterized in that, The device includes: The target image acquisition module is configured to acquire a target image, wherein the target image is a two-dimensional image with target pixel specifications from a target viewpoint; The image generation module is configured to input the target image into a controllable generation model, triggering the controllable generation model to infer the target base color map, target normal map, and target height map based on the target image, respectively. The rendering module is configured to perform texture rendering based on the target base color map, the target normal map and the target height map to obtain a target texture, wherein the target texture is a three-dimensional image from the target's perspective. The controllable generation model is an image generation model trained based on a first sample image, a sample base color map, a sample normal map, and a sample height map. The first sample image has the target pixel specifications and corresponds to the target viewpoint. The sample base color map is used to describe the base color information of the first sample image. The sample normal map is used to describe the normal information of the first sample image, and the normal information is used to represent the surface orientation of each pixel in the first sample image. The sample height map is used to describe the height information of the first sample image, and the height information is used to represent the relative height of each pixel in the first sample image.

9. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the three-dimensional image generation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device performs the three-dimensional image generation method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes a computer program stored in a readable storage medium, wherein at least one processor of a computer device reads from and executes the computer program, causing the device to perform the three-dimensional image generation method as described in any one of claims 1 to 7.

Images