An image generation method, apparatus, device, and storage medium

Abstract

This application discloses an image generation method, apparatus, device, and storage medium, facilitating the generation of images with paper texture effects based on user needs. In this application embodiment, an initial image is obtained, and a pre-trained material model is acquired; paper texture parameters are obtained based on the material model; the paper texture parameters and the initial image are fused to obtain a first fused image. In this application embodiment, a pre-trained material model is used to generate paper texture parameters that can characterize the texture diversity of paper textures. Then, the obtained fingerprint parameters are fused with the initial image, so that the resulting first fused image can produce a paper texture effect of the same material, avoiding the dependence on manual processing in the paper texture effect generation process.

Description

Technical Field

[0001] This invention relates to the field of terminal technology, and in particular to an image generation method, apparatus, device, and storage medium. Background Technology

[0002] In digital painting, to achieve the ultimate realistic painting effect, a paper texture effect is added to the artwork. The paper texture effect refers to an artistic effect created by simulating painting on real-world materials (such as cloth, paper, and walls) using computers. Paintings on such materials need to realistically reflect the effects of the material's physical properties, such as the appearance of the material's texture, color changes caused by lighting, and variations in the rate of pigment deposition.

[0003] In related technologies, the main approach is to manually photograph natural material samples and then use image editing software to manually adjust the photographed sample images to obtain paper texture materials. This method requires a lot of manual operation and the accumulation of artist experience, and the production process is tedious and complicated, making it impossible to quickly generate the required texture materials. Summary of the Invention

[0004] The purpose of this invention is to provide an image generation method, apparatus, device, and storage medium to solve the technical problems in the prior art.

[0005] In a first aspect, embodiments of this application provide an image generation method, the method comprising: acquiring an initial image and acquiring a pre-trained material model; obtaining paper texture parameters based on the material model; and performing a fusion processing on the paper texture parameters and the initial image to obtain a first fused image.

[0006] In this embodiment of the application, a paper texture parameter that can characterize the texture diversity of paper texture is generated by a pre-trained material model. Then, the obtained fingerprint parameter is fused with the initial image so that the first fused image can produce a paper texture effect of the same material, thus avoiding the dependence of the paper texture effect generation process on manual labor.

[0007] In some possible embodiments, obtaining paper texture parameters based on the material model includes: using the acquired user requirements as input to the material model to obtain a material bitmap; inputting the material bitmap into a pre-trained PBR texture generation model to obtain a PBR texture; and obtaining paper texture parameters based on the PBR texture.

[0008] In this embodiment, a PBR texture generation model is used to obtain paper texture parameters, which fully integrates the texture characteristics generated by PBR and the natural texture of physical simulation, making the first fused image more delicate and realistic.

[0009] In some possible embodiments, the acquired user requirements are used as input to the material model, including: acquiring user descriptive words, obtaining a target material reference image based on the user descriptive words, and using the target material reference image as input to the material model; or, acquiring a material reference image uploaded by the user and using the material reference image as input to the material model; or, based on the user's selection operation, determining a target material reference image from a preset set of material reference images and using the target material reference image as input to the material model.

[0010] This application provides multiple input methods to make it more universal.

[0011] In some possible embodiments, the source bitmap is input into a pre-trained PBR texture generation model to obtain a PBR texture, including: inputting the source bitmap into the PBR texture model to obtain a color map, a normal map, a roughness map, and a metallic map; using the color map, normal map, roughness map, and metallic map as PBR textures; wherein, the color map represents the surface color parameters of the target material corresponding to the source bitmap, the normal map represents the surface bump parameters of the target material, the roughness map represents the gloss parameters of the target material surface, and the metallic map represents the metallic parameters of the target material surface.

[0012] In this embodiment, different PBR maps can reflect different material properties, so by setting multiple PBR maps, the resulting first fused image is more realistic.

[0013] In some possible embodiments, the paper texture parameters and the initial image are fused to obtain a first fused image, including: for each pixel in the initial image: determining the normal pixel corresponding to the pixel in the normal map, determining the roughness pixel corresponding to the pixel in the roughness map, and determining the metallic pixel corresponding to the pixel in the metallicity map; obtaining first image parameters based on the paper texture parameters corresponding to the normal pixel, the roughness pixel, and the metallicity pixel respectively; and obtaining the first fused image based on the first image parameters, the initial image, and the paper texture parameters corresponding to the color map.

[0014] In this embodiment of the application, by blending the PBR texture with the initial image, the texture of the material is displayed in the initial image, so that the first blended image can well reflect the effect of the material.

[0015] In some possible embodiments, after obtaining the first fused image, the method further includes: displaying a parameter adjustment interface in response to a user-triggered parameter adjustment interface display operation; determining the parameter to be adjusted and the target parameter value based on the parameter adjustment operation triggered by the user in the parameter adjustment interface; and adjusting the paper texture parameter based on the parameter to be adjusted and the target parameter value to obtain the second fused image.

[0016] In this application embodiment, users can adjust the parameters of the first fused image to make the obtained second fused image meet the different needs of different users, making this application more universal.

[0017] In some possible embodiments, the method further includes: if a user-inputted image descriptor is obtained simultaneously when the initial image is acquired, then the parameter to be adjusted and the target parameter value are determined based on the image descriptor; the paper texture parameter is fused with the initial image to obtain a first fused image, including: adjusting the parameter to be adjusted of the third fused image obtained by fusing the paper texture parameter with the initial image to the target parameter value to obtain the first fused image.

[0018] In this embodiment of the application, the user can input their own requirements when inputting the initial image, so that the resulting first fused image can meet the needs of different users.

[0019] In some possible embodiments, the method further includes: obtaining paper texture parameters based on the acquired pre-trained material model;

[0020] A first paper texture image is obtained based on the paper texture parameters; drawing strokes are obtained based on the drawing operation triggered by the user in the first paper texture image; the first paper texture image and the drawing strokes are fused to obtain a drawing image.

[0021] In this embodiment, users can also create in real time on the first paper texture image provided in this application, which enhances the user experience and fun.

[0022] Secondly, embodiments of this application also provide an image generation apparatus, the apparatus comprising:

[0023] The acquisition module is used to acquire the initial image and the pre-trained source model;

[0024] The paper texture parameter generation module is used to obtain paper texture parameters based on the material model;

[0025] The fusion processing module is used to fuse the paper texture parameters with the initial image to obtain the first fused image.

[0026] In some possible embodiments, the paper texture parameter generation module is specifically used for: taking the acquired user requirements as input to the material model to obtain a material bitmap; inputting the material bitmap into a pre-trained PBR texture generation model to obtain a PBR texture; and obtaining paper texture parameters based on the PBR texture.

[0027] In some possible embodiments, the acquisition module is specifically used to: acquire user descriptive words, obtain a target material reference image based on the user descriptive words, and use the target material reference image as input to the material model; or, acquire a material reference image uploaded by the user and use the material reference image as input to the material model; or, based on the user's selection operation, determine a target material reference image from a preset set of material reference images and use the target material reference image as input to the material model.

[0028] In some possible embodiments, the paper texture parameter generation module is specifically used to: input the material bitmap into the PBR texture model to obtain a color map, a normal map, a roughness map, and a metallic map; and use the color map, normal map, roughness map, and metallic map as PBR textures; wherein, the color map represents the surface color parameters of the target material corresponding to the material bitmap, the normal map represents the surface bump parameters of the target material, the roughness map represents the gloss parameters of the target material surface, and the metallic map represents the metallic parameters of the target material surface.

[0029] In some possible embodiments, the fusion processing module is specifically used for: for each pixel in the initial image: determining the normal pixel corresponding to the pixel in the normal map, determining the roughness pixel corresponding to the pixel in the roughness map, and determining the metallic pixel corresponding to the pixel in the metallicity map; obtaining first image parameters based on the paper texture parameters corresponding to the normal pixel, the roughness pixel, and the metallicity pixel respectively; and obtaining a first fused image based on the first image parameters, the initial image, and the paper texture parameters corresponding to the color map.

[0030] In some possible embodiments, the fusion processing module is further configured to: display a parameter adjustment interface in response to a user-triggered parameter adjustment interface display operation; determine the parameter to be adjusted and the target parameter value based on the user-triggered parameter adjustment operation in the parameter adjustment interface; and adjust the paper texture parameter based on the parameter to be adjusted and the target parameter value to obtain a second fused image.

[0031] In some possible embodiments, the acquisition module is further configured to: if the image descriptive words input by the user are acquired at the same time as the initial image, determine the parameter to be adjusted and the target parameter value based on the image descriptive words; the fusion processing module is further configured to adjust the parameter to be adjusted of the third fused image obtained by fusing the paper texture parameters with the initial image to the target parameter value, thereby obtaining the first fused image.

[0032] In some possible embodiments, the paper texture parameter generation module is further used to obtain paper texture parameters based on the acquired pre-trained material model; the fusion processing module is further used to obtain a first paper texture image based on the paper texture parameters; the acquisition module is further used to acquire drawing strokes based on the drawing operation triggered by the user in the first paper texture image; and the fusion processing module is further used to perform fusion processing on the first paper texture image and the drawing strokes to obtain a drawing image.

[0033] Thirdly, another embodiment of this application also provides an electronic device, including at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the methods provided in the first or second aspect of this application.

[0034] Fourthly, another embodiment of this application provides a computer-readable storage medium storing a computer program for causing a computer to perform any of the methods provided in the first or second aspect of this application.

[0035] Fifthly, another embodiment of this application also provides a computer program product, characterized in that the computer program product includes: computer program code, which, when run on a computer, causes the computer to perform any of the methods provided in the first or second aspect of the embodiments described above.

[0036] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0037] Figure 1 A schematic diagram of a paper texture generation method in the related art provided in the embodiments of this application;

[0038] Figure 2 This is a schematic diagram of the overall process of an image generation method provided in an embodiment of this application;

[0039] Figure 3 A schematic diagram illustrating a user-uploaded initial image as part of an image generation method provided in this application embodiment;

[0040] Figure 4 A schematic diagram illustrating the process of obtaining paper texture parameters based on a material model in an image generation method provided in this application embodiment;

[0041] Figure 5This is a schematic diagram of the input and output of a material model for an image generation method provided in an embodiment of this application;

[0042] Figure 6 A schematic diagram of an uploaded material reference image provided for an image generation method according to an embodiment of this application;

[0043] Figure 7 This is a schematic diagram of the input and output of a generation model for an image generation method provided in an embodiment of this application;

[0044] Figure 8 This is a schematic diagram illustrating the process of fusing paper texture parameters with an initial image in an image generation method provided in an embodiment of this application.

[0045] Figure 9 This is a schematic diagram illustrating the fusion processing of paper texture parameters and an initial image in an image generation method provided in an embodiment of this application.

[0046] Figure 10 This is a schematic flowchart illustrating the process of adjusting image parameters of a first fused image in an image generation method provided in an embodiment of this application.

[0047] Figure 11 A schematic diagram of a parameter adjustment interface for an image generation method provided in an embodiment of this application;

[0048] Figure 12 A schematic diagram of the parameters corresponding to the PBR texture of an image generation method provided in an embodiment of this application;

[0049] Figure 13 A schematic diagram illustrating the process of generating a first fused image based on user requirements using an image generation method provided in this application embodiment;

[0050] Figure 14 A schematic diagram illustrating the process of real-time drawing in a paper texture image according to an embodiment of this application;

[0051] Figure 15 This is a schematic diagram illustrating an image generation method provided in this application, which generates a paper texture image based on a user-uploaded material reference image or language description and renders it in real time based on drawing strokes.

[0052] Figure 16 This is a schematic diagram illustrating a method for generating an image based on a material reference image set, generating a paper texture image, and rendering it in real time based on drawing strokes, as provided in an embodiment of this application.

[0053] Figure 17 A schematic diagram illustrating the generation of a first fused image based on a user-uploaded material reference image or linguistic descriptive words, as provided in an embodiment of this application;

[0054] Figure 18 This is a schematic diagram illustrating the generation of a first fused image from a material reference image set, as provided in an embodiment of this application, using an image generation method.

[0055] Figure 19 This is a schematic diagram of the system architecture of an image generation method provided in an embodiment of this application;

[0056] Figure 20 A schematic diagram of an apparatus for an image generation method provided in an embodiment of this application;

[0057] Figure 21 This is a schematic diagram of an electronic device for an image generation method provided in an embodiment of this application. Detailed Implementation

[0058] The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0059] To better understand the technical solution of this application, the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0060] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0061] The terminology used in the embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. The singular forms “a,” “the,” and “the” used in the embodiments of this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0062] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0063] To facilitate understanding, the terms used in the embodiments of this application will be explained first:

[0064] Paper texture effect refers to an artistic effect created by using computers to simulate painting on real-world materials (such as fabric, paper, and walls).

[0065] Color map (Base Color / Diffuse): Determines the basic color or pattern of an object, reflecting the color of the material surface;

[0066] Normal Map: Creates subtle bumps on a surface to simulate material details such as fabric texture and metal scratches;

[0067] Roughness Map: Controls the smoothness or roughness of a material. Higher roughness makes the object more matte, while lower roughness makes the object more glossy.

[0068] Metallization Map: Defines whether a surface is metallic. Metallic parts typically reflect ambient light, while non-metallic parts appear more matte.

[0069] In digital painting, to achieve the ultimate realistic painting effect, a paper texture effect is added to the artwork. The paper texture effect refers to an artistic effect created by simulating painting on real-world materials (such as cloth, paper, and walls) using computers. Paintings on such materials need to realistically reflect the effects of the material's physical properties, such as the appearance of the material's texture, color changes caused by lighting, and variations in the rate of pigment deposition.

[0070] In related technologies, the main approach is to manually photograph natural material samples and generate paper texture materials through image processing, such as... Figure 1 As shown, the process begins with photographing physical samples of various natural materials, such as paper, fabric, and wood grain. Image editing software is then used to adjust the contrast, add highlights, separate shadow areas, and sharpen the original samples. Based on experience, different texture layers are blended and overlaid to enhance the visual texture. Layer parameters are repeatedly adjusted, and the scale, direction, and color of the textures are manually optimized to ultimately generate texture materials that meet the design requirements.

[0071] However, the above-mentioned manual collection method requires a lot of manual operation and artist experience accumulation. The production process is tedious and complicated, and it cannot quickly generate the required texture materials. Limited by the number of preset textures, it is difficult to meet the personalized needs of different users for diverse texture styles. It is necessary to rely on official support to obtain the required content. This solution relies on manually collected physical samples that can be photographed, and then goes through tedious post-processing to complete the creation of the canvas's realistic effect. This kind of mixed-processed texture image has poor performance in terms of physical properties such as light and reflection, and it is difficult to accurately simulate the delicate texture of natural materials.

[0072] To address the aforementioned problems, embodiments of this application provide an image generation method, apparatus, device, and storage medium to solve these problems. The inventive concept of this application can be summarized as follows: acquiring an initial image and acquiring a pre-trained material model; obtaining paper texture parameters based on the material model; and fusing the paper texture parameters with the initial image to obtain a first fused image.

[0073] In this embodiment of the application, a paper texture parameter that can characterize the texture diversity of paper texture is generated by a pre-trained material model. Then, the obtained fingerprint parameter is fused with the initial image so that the first fused image can produce a paper texture effect of the same material, thus avoiding the dependence of the paper texture effect generation process on manual labor.

[0074] For ease of understanding, the image generation method provided in this application embodiment will be described in detail below with reference to the accompanying drawings:

[0075] like Figure 2 The diagram shown is a schematic representation of the overall process of an image generation method provided in an embodiment of this application, wherein:

[0076] In step 201: Obtain the initial image and the pre-trained material model.

[0077] In this embodiment of the application, the initial image obtained can be an image uploaded by the user. This initial image is an image without paper texture effect. After receiving the initial image uploaded by the user, it is necessary to upload the initial image to the material model so as to obtain the paper texture parameters.

[0078] In some possible embodiments, the user can upload the initial image through an image upload interface displayed on the terminal device, for example: Figure 3 As shown, the user first clicks the plus sign in the image upload interface, and then shows the user the option to take an image or upload from the album. If the user selects "take an image", the camera on the terminal device is invoked so that the user can take an image. If the user selects "select from album", the gallery on the terminal device is invoked so that the user can select an initial image from the gallery.

[0079] In step 202: the paper texture parameters are obtained based on the material model.

[0080] In this embodiment of the application, the material model is used to generate a material bitmap associated with the input image based on the image input at the input terminal, and then obtain the paper texture parameters based on the material bitmap.

[0081] In some possible embodiments, paper texture parameters are obtained based on the material model, specifically as follows: Figure 4 The steps shown are as follows:

[0082] In step 401: the obtained user requirements are used as input to the material model to obtain the material bitmap.

[0083] In this embodiment, the material model can be a generative model from related technologies that has been trained using the material training samples provided in this embodiment. The process of training the generative model using material training samples is the same as the training method for generative models in related technologies, and will not be described again here. During the training process, such as... Figure 5 As shown, a certain degree of randomness occurs during the sampling process. This randomness can characterize the microscopic changes in the texture. Therefore, the input of the generation model set in this application includes not only the material reference image corresponding to the material, but also the random noise image obtained by sampling through a random Gaussian distribution. By using the random noise image and the material reference image as inputs to the generation model, the generated material bitmap can be a high-quality digital texture material that meets the user's needs.

[0084] The material training samples provided in this application embodiment can be pre-constructed based on the material reference images corresponding to various materials. In order to ensure that the obtained material model can meet the needs of different users, it is necessary to cover the material reference images corresponding to various common materials as much as possible when constructing the material training samples, so as to ensure that the obtained material model is more universal.

[0085] In some possible embodiments, the acquired user requirements are used as input to the material model, which can be implemented in any of the following ways: acquiring user descriptive words, obtaining a target material reference image based on the user descriptive words, and using the target material reference image as input to the material model; or, acquiring a material reference image uploaded by the user and using the material reference image as input to the material model; or, based on the user's selection operation, determining a target material reference image from a preset set of material reference images and using the target material reference image as input to the material model.

[0086] In this embodiment of the application, users can describe their needs in natural language, and can also use images of real materials in nature captured by image acquisition devices such as cameras or video cameras as input to the material model. They can also select images that meet their needs from the material reference image set, which includes a variety of reference images of different material types, including but not limited to: mahogany, Xuan paper, oil painting paper, cement wall, linen, etc.

[0087] For example: Figure 6As shown, if a user enters the text "Generate an image with an oil painting paper effect" in the input box displayed on the terminal device, the user can obtain the corresponding material reference image for oil painting paper from the material reference image set and use it as the input for the material model; alternatively, if the user has saved the material reference image for oil painting paper taken from a local folder, the user can upload the corresponding material reference image from the local folder and use the uploaded material reference image for the material model; or, if the user clicks the material reference image set option, the user will be shown the material reference images corresponding to each material, and the user can select the image that meets their needs from the displayed material reference images as the input for the material model.

[0088] In step 402: Input the source bitmap into the pre-trained PBR texture generation model to obtain the PBR texture.

[0089] In this embodiment, considering that Physically Based Rendering (PBR) is a rendering technique for achieving realism in 3D graphics, PBR simulates the interaction between light and materials based on physical principles. PBR accurately reproduces the reflection, refraction, and scattering of light through mathematical models, enabling objects to exhibit consistent and realistic effects under different lighting conditions. PBR textures are an important component of PBR; by defining different types of PBR textures, different properties of materials under lighting conditions can be reflected. Therefore, this application constructs a PBR texture generation model for generating PBR textures. Specifically, a pre-built texture training sample set can be used to train the generation model, and the converged generation model is used as the PBR texture generation model. During the training process, as follows... Figure 7 As shown, the input to the generative model is a source bitmap and a random noise map obtained by sampling through a random Gaussian distribution. The output is a PBR map, which includes, but is not limited to, color maps, normal maps, roughness maps, and metallic maps.

[0090] In some possible embodiments, the source bitmap is input into a pre-trained PBR texture generation model to obtain a PBR texture. Specifically, this can be implemented as follows: the source bitmap is input into the PBR texture model to obtain a color texture, a normal texture, a roughness texture, and a metallic texture; the color texture, normal texture, roughness texture, and metallic texture are used as PBR textures.

[0091] In the embodiments of this application, the color map represents the surface color parameters of the target material corresponding to the material bitmap, the normal map represents the surface bump parameters of the target material, the roughness map represents the gloss parameters of the target material surface, and the metallicity map represents the metallic parameters of the target material surface.

[0092] In step 403: paper texture parameters are obtained based on PBR mapping.

[0093] In this embodiment, the color parameters corresponding to the color map, the surface bump parameters corresponding to the normal map, the gloss parameters corresponding to the roughness map, and the metal parameters corresponding to the metallicity map are used as paper texture parameters.

[0094] In step 203: The paper texture parameters and the initial image are fused to obtain the first fused image.

[0095] In this embodiment of the application, after obtaining the paper texture parameters, the paper texture parameters and the initial image without paper texture can be fused together to obtain a first fused image with paper texture.

[0096] In some possible embodiments, the paper texture parameters are fused with the initial image to obtain a first fused image. For each pixel in the initial image, specific implementations may be performed as follows: Figure 8 The steps shown are used to perform the fusion process, wherein:

[0097] In step 801: Determine the normal pixel corresponding to the pixel in the normal map, determine the roughness pixel corresponding to the pixel in the roughness map, and determine the metallic pixel corresponding to the pixel in the metallicity map.

[0098] In this embodiment of the application, after obtaining each PBR map, it is necessary to determine the pixel points corresponding to the normal map, roughness map and metallicity map, where the corresponding pixel points are the pixels in the PBR map that have the same position as the pixels in the initial image; then the paper texture parameters corresponding to the pixels with the same position are fused to obtain the reflection of light on the surface of the initial image (first image parameters).

[0099] In step 802: the first image parameters are obtained based on the paper texture parameters corresponding to the normal pixels, roughness pixels, and metallicity pixels, respectively.

[0100] In the embodiments of this application, such as Figure 9 As shown, when performing fusion processing on each pixel, the bidirectional reflection distribution function (i.e., Formula 1) can first be used to process the normal map, roughness map and metallic map in the PBR map to obtain the reflection of light on the surface of the initial image (first image parameters).

[0101]

[0102] Among them, f rHere, n is the image parameter corresponding to the normal pixel (i.e., surface bump parameter), α is the image parameter corresponding to the roughness pixel (i.e., gloss parameter), F0 is the image parameter corresponding to the metallicity pixel (i.e., metallic parameter), l is the preset light direction, v is the preset user's viewing direction, and h is the half-angle vector, which can be calculated using formula 2.

[0103]

[0104] Where h is a half-angle vector, l is a preset ray direction, and v is a preset user's line of sight direction.

[0105] In step 803: the first fused image is obtained based on the first image parameters, the initial image, and the paper texture parameters corresponding to the color map.

[0106] In this embodiment of the application, after obtaining the first image parameters, the first image parameters, the initial image, and the color map can be fused using the bidirectional reflection distribution function image rendering principle to obtain the first fused image.

[0107] In other possible embodiments, different users may have different needs or preferences. Therefore, even if the initial input image is the same and the selected material type is the same, the expected first fused image may not be the same. In order to make the image generation method provided in this application embodiment more universal, after obtaining the first fused image, it can also be implemented as follows: Figure 10 The steps shown are used to adjust the image parameters of the first fused image, wherein:

[0108] In step 1001: In response to the user-triggered parameter adjustment interface display operation, the parameter adjustment interface is displayed.

[0109] In this embodiment, after obtaining the first fused image, the user can adjust the image parameters of the first fused image, thereby changing the effect of the first fused image to meet the user's needs. The image parameters in the parameter adjustment interface include, but are not limited to: paper texture opacity, gloss absorption, shadow, pigment deposition, and scale.

[0110] For example: In the upper left corner of the first fused image display interface, there is a parameter adjustment button. If the user clicks the parameter adjustment button, it can be determined that the user has triggered the parameter adjustment interface display operation, and the following will be displayed: Figure 11 The parameter adjustment interface shown.

[0111] In step 1002: the parameter to be adjusted and the target parameter value are determined based on the parameter adjustment operation triggered by the user in the parameter adjustment interface.

[0112] In this embodiment of the application, the user can adjust the image parameters of the first fused image in the parameter adjustment interface.

[0113] For example: the parameter adjustment interface is as follows Figure 11 As shown, the value of each image parameter in the parameter adjustment interface is the value of the image parameter corresponding to the first paper texture parameter. Users can adjust the image parameters by dragging the adjustment bar corresponding to the image parameter.

[0114] Understandable Figure 11 The parameter adjustment interface shown and the image parameter adjustment method described above are only one embodiment and are not intended to limit the style of the parameter adjustment interface or the method of image parameter adjustment. In specific implementations, the style of the parameter adjustment interface and the method of image parameter adjustment can be set according to the requirements.

[0115] In step 1003: The paper texture parameters are adjusted based on the parameters to be adjusted and the target parameter values ​​to obtain the second fused image.

[0116] In this embodiment, since the first fused image is obtained by fusing the PBR map and the initial image, after obtaining the values ​​of the parameters to be adjusted and the target parameters, it is necessary to map the parameters to be adjusted onto the PBR map, such as... Figure 12 As shown, the effect of the first fused image is changed by adjusting the parameters corresponding to each PBR map.

[0117] In some possible embodiments, in order for the obtained first fused image to reflect the lighting conditions, the first fused image can be rendered after obtaining it by combining a pre-set lighting direction and / or camera angle.

[0118] In other possible embodiments, in addition to adopting Figure 10 The method shown allows for adjustments to the first fused image after its generation. Alternatively, specific requirements can be input using natural language descriptions while simultaneously inputting the initial image. This ensures that the resulting first fused image directly meets the user's specific needs. Specifically, this can be implemented as follows: Figure 13 The method shown, wherein:

[0119] In step 1301: obtain the initial image, the image description words input by the user, and the pre-trained material model.

[0120] In this embodiment of the application, the steps of obtaining the initial image and obtaining the material model are the same as those in step 201 above, and will not be described again here.

[0121] In some possible embodiments, users can input image descriptive terms via voice or text boxes. This application does not limit the specific implementation of the user descriptive term input method. The specific style of the image descriptive terms can be input by the user according to their needs.

[0122] In step 1302: the paper texture parameters are obtained based on the material model.

[0123] In this embodiment of the application, the specific implementation of this step is the same as that of step 201, and will not be repeated here.

[0124] In step 1303: Determine the parameters to be adjusted and the target parameter values ​​based on the image descriptive terms.

[0125] In this embodiment of the application, in order to adjust the image parameters of the third fused image according to the image descriptive words input by the user, it is necessary to convert the image descriptive words input by the user into image parameters.

[0126] In some possible embodiments, converting image descriptors into image parameters can be specifically implemented by: determining keywords in the image descriptors, and obtaining target image parameters and the range of the target image parameters based on the keywords.

[0127] In this embodiment of the application, the parameter range corresponding to each image parameter can be preset, and each parameter range can be associated with a preset set of keywords, so that the image parameters that need to be adjusted can be obtained according to the keywords in the image description words input by the user.

[0128] For example, regarding gloss absorption in an image parameter set, the corresponding parameter ranges are set to 0%-30%, 30%-60%, and 60%-100%. For the 0%-30% range, the keyword set includes: "darker," "darker," and "lighter." If the user inputs the image description: "darker the image," then the keyword is determined to be "darker," and the image parameter to be adjusted is gloss absorption, with a corresponding range of 0%-30%. The midpoint of this range is then used as the target parameter value.

[0129] In step 1304: The paper texture parameters and the initial image are fused to obtain the third fused image.

[0130] In this embodiment of the application, the specific implementation of this step is the same as that of step 301, and will not be described again here.

[0131] In step 1305: The paper texture parameters of the third fused image are adjusted based on the parameters to be adjusted and the target parameter values ​​to obtain the first fused image.

[0132] In this embodiment of the application, the specific implementation of this step is the same as that of step 1003, and will not be described again here.

[0133] In other possible embodiments, the user may not have an initial image that requires fusion processing, but instead wants to paint or write on the target material in real time. In this case, to meet the user's needs, the following implementation can be adopted: Figure 14 The steps shown are as follows:

[0134] In step 1401: the paper texture parameters are obtained based on the pre-trained material model.

[0135] In this embodiment of the application, the specific implementation of this step is the same as step 202 described above, and will not be repeated here.

[0136] In step 1402: the first paper texture image is obtained based on the paper texture parameters.

[0137] In this embodiment, after obtaining the PBR texture, it is not necessary to fuse the PBR texture with the initial image; the first paper texture image can be obtained by fusing the individual PBR textures. Other steps and methods are the same as described above. Figure 8 The method shown is the same, so it will not be repeated here.

[0138] In step 1403: Based on the drawing operation triggered by the user in the first paper texture image, the drawing strokes are obtained.

[0139] In step 1404: the first paper texture image and the drawing strokes are fused to obtain the drawing image.

[0140] In this embodiment, the drawing strokes are used as the initial image, and the first paper texture image is used as the input to the material model. The specific implementation method is similar to... Figure 2 The same applies, so I will not repeat it here.

[0141] To facilitate a further understanding of the image generation method provided in this application, the following description, in conjunction with specific application scenarios and related accompanying drawings, will further illustrate the image generation method provided in this application:

[0142] like Figure 15 As shown, if the user enters "Generate an oil painting paper for me" in the text box or uploads a material reference image of the oil painting paper from a local folder, but does not upload an initial image, then the above will be used. Figure 14 The steps shown generate a paper texture image corresponding to the painting paper. If the user paints on the paper texture image, the user's brushstrokes are captured in real time and then processed according to the above... Figure 14Steps 1403 and 1404 in the process merge the user's drawing strokes with the corresponding paper texture image of the oil painting paper and display it to the user.

[0143] like Figure 16 As shown, the user selected a material reference image for the oil painting paper from a pre-set set of material reference images, but did not upload an initial image. Therefore, through the above... Figure 14 The steps shown generate a paper texture image corresponding to the painting paper. If the user paints on the paper texture image, the user's brushstrokes are captured in real time and then processed according to the above... Figure 14 Steps 1403 and 1404 in the process merge the user's drawing strokes with the corresponding paper texture image of the oil painting paper and display it to the user.

[0144] like Figure 17 As shown, if the user enters "Give me a canvas" in the text box or uploads a material reference image of the canvas from a local folder, and uploads an initial image, then the above will be successfully generated. Figure 2 The steps shown are used to add a paper texture effect corresponding to oil painting paper to the initial image to obtain a first blended image that meets the user's needs.

[0145] like Figure 18 As shown, the user selected the material reference image corresponding to the oil painting paper from the pre-set material reference image set and uploaded the initial image. Then, through the above... Figure 2 The steps shown are used to add a paper texture effect corresponding to oil painting paper to the initial image to obtain a first blended image that meets the user's needs.

[0146] Based on the same inventive concept, the system architecture of an image generation method provided in the embodiments of this application is described below, such as... Figure 19 As shown, where:

[0147] First, the acquired user requirements (user descriptions, uploaded material reference images, and selections from the material reference image set) are used as input to the material model, resulting in a material bitmap output by the material model. The material bitmap is then used as input to the PBR texture generation model, resulting in a PBR texture output by the PBR texture generation model. Based on the PBR texture, paper texture parameters are obtained. A first blended image is generated based on the user-uploaded initial image and the paper texture parameters. The first blended image is then rendered using a preset camera angle and / or lighting direction. If the user has specific requirements for the generated first blended image, the paper texture parameters of the rendered first blended image can be adjusted. Alternatively, the user can directly input their requirements when entering the user description, adjust the parameters of the first fingerprint image after obtaining the first blended image, and then proceed with rendering.

[0148] Based on the same inventive concept, embodiments of this application also provide an image generation apparatus, such as... Figure 20 As shown, the device includes:

[0149] The acquisition module is used to perform the above steps 201, 1301 and 1403;

[0150] The paper texture parameter generation module is used to execute the above steps 202, 401, 402, 403, 1302, 1303 and 1401;

[0151] The fusion processing module is used to execute the above steps 203, 801, 802, 803, 1001, 1002, 1003, 1304, 1305, 1402 and 1404.

[0152] Corresponding to the above embodiments, this application also provides an electronic device. Figure 21 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. The electronic device 2100 may include a processor 2101, a memory 2102, and a communication unit 2103. These components communicate through one or more buses. Those skilled in the art will understand that the structure of the electronic device shown in the figure does not constitute a limitation on the embodiment of the present invention. It may be a bus topology or a star topology, and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0153] The communication unit 2103 is used to establish a communication channel, enabling the electronic device to communicate with other devices. It receives user data from other devices or sends user data to other devices.

[0154] The processor 2101 serves as the control center of the electronic device, connecting various parts of the device via interfaces and lines. It executes software programs and / or modules stored in the memory 2102, and calls data stored in the memory to perform various functions and / or process data. The processor can be composed of integrated circuits (ICs), such as a single packaged IC or multiple packaged ICs with the same or different functions connected together. For example, the processor 2101 may consist only of a central processing unit (CPU). In this embodiment, the CPU may have a single processing core or include multiple processing cores.

[0155] The memory 2102 is used to store the execution instructions of the processor 2101. The memory 2102 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk.

[0156] When the execution instructions in memory 2102 are executed by processor 2101, the electronic device 2100 is able to perform operations. Figure 2 Some or all of the steps in the illustrated embodiments.

[0157] In a specific implementation, the present invention also provides a computer storage medium, wherein the computer storage medium may store a program, which, when executed, may include some or all of the steps of the various embodiments of the image generation method provided by the present invention. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0158] In some possible implementations, various aspects of the terminal device control method provided in this application can also be implemented in the form of a program product, which includes program code. When the program product is run on a computer device, the program code is used to cause the computer device to perform the steps in the terminal device control method according to the various exemplary embodiments of this application described above.

[0159] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0160] The program product for controlling a terminal device according to embodiments of this application may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on an electronic device. However, the program product of this application is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0161] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.

[0162] The same or similar parts between the various embodiments in this specification can be referred to mutually. In particular, the device embodiments and terminal embodiments are basically similar to the method embodiments, so the description is relatively simple, and the relevant parts can be referred to the description in the method embodiments.

Claims

1. An image generation method, characterized in that, The method includes: Obtain the initial image and the pre-trained source model; The paper texture parameters are obtained based on the material model; The paper texture parameters are fused with the initial image to obtain a first fused image.

2. The method according to claim 1, characterized in that, The process of obtaining paper texture parameters based on the material model includes: The obtained user requirements are used as input to the material model to obtain the material bitmap; The source bitmap is input into a pre-trained PBR texture generation model to obtain a PBR texture; The paper texture parameters are obtained based on the PBR texture.

3. The method according to claim 2, characterized in that, The step of using the acquired user requirements as input to the material model includes: Obtain user descriptive terms, obtain a target material reference image based on the user descriptive terms, and use the target material reference image as input to the material model; or, Obtain the material reference image uploaded by the user, and use the material reference image as input to the material model; or, Based on the user's selection, a target material reference image is determined from a preset set of material reference images, and the target material reference image is used as the input to the material model.

4. The method according to claim 2, characterized in that, The step of inputting the source bitmap into a pre-trained PBR texture generation model to obtain a PBR texture includes: The bitmap image is input into the PBR texture model to obtain color texture, normal texture, roughness texture and metallic texture; The color map, the normal map, the roughness map, and the metallic map are used as the PBR map; Wherein, the color map represents the surface color parameters of the target material corresponding to the material bitmap, the normal map represents the surface bump parameters of the target material, the roughness map represents the gloss parameters of the target material surface, and the metallicity map represents the metallic parameters of the target material surface.

5. The method according to claim 4, characterized in that, The step of fusing the paper texture parameters with the initial image to obtain a first fused image includes: For each pixel in the initial image: Determine the normal pixel corresponding to the pixel in the normal map, determine the roughness pixel corresponding to the pixel in the roughness map, and determine the metallic pixel corresponding to the pixel in the metallicity map. The first image parameters are obtained based on the paper texture parameters corresponding to the normal pixels, the roughness pixels, and the metallicity pixels, respectively. The first fused image is obtained based on the first image parameters, the initial image, and the paper texture parameters corresponding to the color map.

6. The method according to claim 1, characterized in that, After obtaining the first fused image, the method further includes: In response to user-triggered parameter adjustment interface display, the parameter adjustment interface is displayed; The parameter to be adjusted and the target parameter value are determined based on the parameter adjustment operation triggered by the user in the parameter adjustment interface. The paper texture parameters are adjusted based on the parameters to be adjusted and the target parameter values ​​to obtain a second fused image.

7. The method according to claim 1, characterized in that, The method further includes: If, when acquiring the initial image, a user-inputted image description is also acquired, then the parameter to be adjusted and the target parameter value are determined based on the image description. The step of fusing the paper texture parameters with the initial image to obtain a first fused image includes: The parameters to be adjusted in the third fused image obtained by fusing the paper texture parameters with the initial image are adjusted to the target parameter value to obtain the first fused image.

8. The method according to claim 1, characterized in that, The method further includes: The paper texture parameters are obtained based on the pre-trained material model. A first paper texture image is obtained based on the paper texture parameters; Based on the drawing operation triggered by the user in the first paper texture image, the drawing strokes are obtained; The first paper texture image and the drawing strokes are fused together to obtain a drawing image.

9. An image generation apparatus, characterized in that, The device includes: The acquisition module is used to acquire the initial image and the pre-trained source model; The paper texture parameter generation module is used to obtain paper texture parameters based on the material model. The fusion processing module is used to fuse the paper texture parameters with the initial image to obtain a first fused image.

10. An electronic device, characterized in that, It includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the electronic device is triggered to perform the method of any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device on which the computer-readable storage medium is located to perform the method according to any one of claims 1-8.

12. A computer program product, characterized in that, The computer program product includes: computer program code, which, when run on a computer, causes the computer to perform the method described in any one of claims 1-8.

Images