Image generation method, device, equipment and medium
By using the simulation model of the target object in different states to render, high-quality sample images are generated for model training, which solves the problems of limited style conversion types and poor model training effects in the existing technology, and realizes personalized image style The transformation is enriched and the display effect is improved.
Patent Information
- Application Number
- CN202110608977.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-01
- Publication Date
- 2024-11-12
- Estimated Expiration
- 2041-06-01
AI Technical Summary
In existing video interactive applications, the style conversion types are limited, which cannot meet the user's personalized image style conversion needs, and the image style conversion model training effect is not good.
By obtaining the initial image of the target object, the image style conversion is performed using a machine learning model based on the simulation model to generate the target image. The simulation model uses the sample images rendered by the rendering parameters in different states as training data to ensure the quality of the sample data and the model training effect.
It enriches the types of image style conversion, improves the display effect of personalized image style conversion, and ensures model training effect and image generation quality.
Smart Images

Figure CN115439305B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to an image generation method, device, equipment and medium. Background Art
[0002] At present, as the functions of video interactive applications are gradually enriched, image style conversion has become a new and interesting way to play. Image style conversion refers to converting the style of one or more images to generate style images that meet user needs.
[0003] However, the types of style conversion supported in existing video interactive applications are still limited and cannot meet the personalized image style conversion needs of users. In addition, the display effect of image style conversion is directly related to the training effect of the model. Therefore, how to ensure the training effect of the model used to achieve personalized image style conversion is also a problem to be solved. Summary of the invention
[0004] In order to solve the above technical problems or at least partially solve the above technical problems, the embodiments of the present disclosure provide an image generation method, apparatus, device and medium.
[0005] In a first aspect, an embodiment of the present disclosure provides an image generation method, comprising:
[0006] Acquire an initial image of the target object, where the initial image is an image of a first image style;
[0007] By inputting the initial image into a first machine learning model, a target image of the target object is obtained based on an output result of the first machine learning model, wherein the target image is an image of the second image style;
[0008] Wherein, the first machine learning model is obtained based on the second machine learning model; the second machine learning model is obtained by training based on the first sample image of the target object and the second sample image of the target object; the first sample image is an image obtained by rendering the simulation model of the target object in different states based on the rendering parameters under the first image style; and the second sample image is an image obtained by rendering the simulation model of the target object in different states based on the rendering parameters under the second image style;
[0009] The simulation model is used to construct graphics of the target object in different states, where the different states include different shapes of the target object and / or different postures of the target object.
[0010] In a second aspect, the present disclosure also provides an image generating device, including:
[0011] An initial image acquisition module, used to acquire an initial image of the target object, wherein the initial image is an image of a first image style;
[0012] a target image determination module, configured to obtain a target image of the target object based on an output result of the first machine learning model by inputting the initial image into a first machine learning model, wherein the target image is an image of a second image style;
[0013] Wherein, the first machine learning model is obtained based on the second machine learning model; the second machine learning model is obtained by training based on the first sample image of the target object and the second sample image of the target object; the first sample image is an image obtained by rendering the simulation model of the target object in different states based on the rendering parameters under the first image style; and the second sample image is an image obtained by rendering the simulation model of the target object in different states based on the rendering parameters under the second image style;
[0014] The simulation model is used to construct graphics of the target object in different states, where the different states include different shapes of the target object and / or different postures of the target object.
[0015] In a third aspect, an embodiment of the present disclosure further provides an electronic device, comprising a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device implements any image generation method provided by the embodiments of the present disclosure.
[0016] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a computing device, the computing device implements any of the image generation methods provided by the embodiments of the present disclosure.
[0017] Compared with the prior art, the technical solution provided by the embodiments of the present disclosure has at least the following advantages:
[0018] In the embodiments of the present disclosure, by utilizing simulation models of the target object in different states and based on different image style requirements, each simulation model is rendered differently to obtain sample images presenting different image styles as paired sample data for model training. Since each pair of sample images is rendered based on the same simulation model, the simulation model can accurately characterize the characteristics of the target object, thereby ensuring the image quality of the sample images. Furthermore, high-quality sample data ensures the model training effect, and a higher model training effect ensures a personalized image style conversion effect on the target object. Therefore, the embodiments of the present disclosure not only enrich the types of image style conversion, but also ensure the model training effect and the display effect of personalized image style conversion. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0020] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0021] Figure 1 A flowchart of an image generation method provided by an embodiment of the present disclosure;
[0022] Figure 2 A schematic diagram of a 3D model of a hand provided in an embodiment of the present disclosure;
[0023] Figure 3 A schematic diagram of a hand image presenting different image styles provided by an embodiment of the present disclosure;
[0024] Figure 4 A schematic diagram of the structure of an image generating device provided by an embodiment of the present disclosure;
[0025] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION
[0026] In order to more clearly understand the above-mentioned objectives, features and advantages of the present disclosure, the scheme of the present disclosure will be further described below. It should be noted that the embodiments of the present disclosure and the features in the embodiments can be combined with each other without conflict.
[0027] In the following description, many specific details are set forth to facilitate a full understanding of the present disclosure, but the present disclosure may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present disclosure, rather than all of the embodiments.
[0028] Figure 1 A flowchart of an image generation method provided in an embodiment of the present disclosure, the method can be executed by an image generation device, the device can be implemented using software and / or hardware, and can be integrated on any electronic device with computing capabilities, such as a mobile phone, notebook, tablet computer, etc.
[0029] Moreover, the image generating device may be implemented in the form of an independent application or a small program integrated on a public platform, or may be implemented as a functional module integrated in an application or small program with an image generating function. The application or small program may include but is not limited to a video interaction application or a video interaction small program, etc.
[0030] like Figure 1 As shown, the image generation method provided by the embodiment of the present disclosure may include:
[0031] S101. Acquire an initial image of a target object, where the initial image is an image of a first image style.
[0032] The target object may be any object with a style change requirement, such as a human or animal limb (specifically including but not limited to hands or feet, and for pets such as cats and dogs, the feet may include claws), or a component of a physical entity, etc. Exemplarily, the electronic device may obtain an initial image of the target object based on an image capture operation or an image upload operation by the user. Taking the target object as a hand as an example, the electronic device may call a camera to capture an image of the user's hand as the initial image according to a photo request triggered by the user.
[0033] S102. Inputting the initial image into the first machine learning model, obtaining a target image of the target object based on an output result of the first machine learning model, wherein the target image is an image of the second image style.
[0034] Among them, the first machine learning model is obtained based on the second machine learning model; the second machine learning model is trained based on the first sample image of the target object (for example, it can be a realistic image, or a real shot image of the target object) and the second sample image of the target object (for example, it can be an image of any style different from the realistic image), the first sample image is an image obtained by rendering the simulation model of the target object in different states based on the rendering parameters under the first image style, and the second sample image is an image obtained by rendering the simulation model of the target object in different states based on the rendering parameters under the second image style; the simulation model is used to construct graphics of the target object in different states, and the different states include different shapes of the target object and / or different postures of the target object. The simulation model can be a pre-built model that can accurately characterize the characteristics of the target object, for example, it can be a 2D model (or a two-dimensional modeling model) or a 3D model (or a 3Dmesh model or a three-dimensional modeling model). The simulation model can be obtained based on an open source simulation model library, or it can be obtained by collecting data required for reconstruction of the simulation model. In order to ensure the sufficiency and diversity of paired sample data used for model training, it is necessary to obtain sample images in multiple states for the same target object to form a large amount of paired sample data. The different shapes of the target object may include the length, width or fatness of the target object, and the fatness may include thin, standard, fat, etc. The specific classification may be determined according to the distinction of the simulation model.
[0035] Taking the target object as a human hand as an example, the hand posture may include various hand gestures, such as hand straightening, finger bending, OK gesture, etc. The hand shape includes the fatness of the hand, such as thin hands (for example, palm thickness is less than a thickness threshold, and the thickness threshold can be flexibly set), standard hands (for example, palm thickness is equal to a thickness threshold), fat hands (for example, palm thickness is greater than a thickness threshold), etc. It should be understood that the fatness of the hand can also be measured according to other available hand features, such as being distinguished according to the diameter of a specific finger, etc.
[0036] In an optional embodiment, the simulation model of the target object in different states can be obtained by adjusting the state parameters (including shape parameters and posture parameters) of the initial simulation model of the target object. The initial simulation model can be obtained based on an open source model library, or it can be obtained by self-modeling. Taking the target object as a limb part as an example, the initial simulation model can be a model of the limb part in a specific standard state. After the user opens the initial simulation model using a modeling tool, the state parameters can be set. The electronic device adjusts the state parameters of the initial simulation model according to the user's parameter setting operation, thereby obtaining multiple simulation models of the limb part in different states. The state parameters can be determined according to the parameter layout in the modeling tool.
[0037] Figure 2 Taking the target object as a hand as an example, a schematic diagram of a 3D model of the hand is shown. The 3D model can be used to exemplify the initial simulation model or simulation model in the embodiment of the present disclosure, but should not be understood as a specific limitation on the embodiment of the present disclosure.
[0038] In an embodiment of the present disclosure, the first machine learning model has the function of generating an image of the second image style. The first machine learning model and the second machine learning model may be machine learning models with the same model structure. In this case, the first machine learning model is obtained based on the second machine learning model, and it can be considered that the model output effect of the first machine learning model is better than the model output effect of the second machine learning model, or the first machine learning model can be considered to be the second machine learning model. The first machine learning model and the second machine learning model may also be machine learning models with different model structures. For example, the first machine learning model is trained based on a third sample image of the target object and a fourth sample image of the target object; the third sample image is an image of the first image style, and the fourth sample image is an image output by the second machine learning model obtained by inputting the third sample image into the second machine learning model.
[0039] Exemplarily, the second machine learning model can be obtained by training a generative adversarial network (GAN) based on a first sample image of the target object and a second sample image of the target object, and the first machine learning model can be obtained by training a conditional generative adversarial network (CGAN) based on a third sample image of the target object and a fourth sample image corresponding to the third sample image output by the second machine learning model. It should be noted that the third sample image and the first sample image have the same image style. During the model training process, the two can be the same image or different images, which is not specifically limited in the embodiments of the present disclosure. In addition, during the model training process, any other applicable neural network can be used, not limited to the above-mentioned generative adversarial network and conditional generative adversarial network.
[0040] For the two-step model training process, on the basis of ensuring the quality of style image generation, the space occupancy and computational complexity of the first machine learning model can be smaller than the model features corresponding to the second machine learning model, so that it can be sent to the user terminal for operation, achieving the effect of taking into account user terminals with different performances. In addition, taking the first sample image as an example of a realistic image, since the first sample image and the second sample image of the target object are both rendered based on the simulation model of the target object, there will be certain differences from the image of the target object in the real scene. By using the second machine learning model to generate a fourth sample image corresponding to the third sample image (of the same style as the first sample image), and then using the third sample image and the fourth sample image to train the first machine learning model, the first machine learning model can have the ability to accurately process the real shot image of the target object, which helps to improve the model training effect, and then helps to improve the style conversion effect for the real shot image.
[0041] The disclosed embodiment utilizes a pre-trained first machine learning model to obtain a style image corresponding to an initial image of a target object, thereby enriching the types of image style conversion, ensuring the display effect of personalized image style conversion, and improving user experience.
[0042] In the embodiment of the present disclosure, considering that more sample data is needed in the model training process, the embodiment of the present disclosure can construct paired sample data by using a simulation model with the ability to represent the characteristics of the target object. On the one hand, the efficiency of generating a variety of paired sample data can be improved by means of the flexible adjustability of the simulation model and the flexible setting of rendering parameters, so as to overcome the problem of insufficient sample data for model training due to the limited number of images of the target object in the first image style and the second image style, especially when the first image style corresponds to the realistic image style, the amount of sample data is very limited; on the other hand, if the simulation model is dynamically reconstructed based on multiple sample images of the first image style of the target object (for example, sample images with the realistic image style) For example, 3D model reconstruction), and based on the currently reconstructed simulation model, a sample image of the required second image style is generated. Since the implementation process of model reconstruction is complicated and the reconstruction process easily introduces more errors, the simulation model constructed multiple times based on multiple sample images of the first image style of the target object has a large deviation in the feature representation of the target object, which in turn causes the sample image of the second image style generated based on the dynamically constructed simulation model to have a large feature deviation from the sample image of the target object in the first image style. That is, at this time, the quality of the paired sample data formed by the sample image of the first image style of the target object and the sample image of the second image style generated based on the model reconstruction is poor (or the data accuracy is low), which affects the model training effect. In the disclosed embodiment, for each predetermined simulation model of the target object, rendering is performed based on rendering parameters under different image styles, and the feature deviation between the obtained first sample image and the second sample image is small, which can ensure the quality of the paired sample data used for model training and ensure the training effect of the image generation model.
[0043] In an optional embodiment, during the sample image construction process, the rendering parameters under different image styles mentioned in the embodiments of the present disclosure include at least one of the following: texture parameters, color parameters, lighting parameters and special effect parameters of the target object; at least one of the rendering parameters under the first image style and the second image style is different, that is, two images corresponding to different image styles can be rendered based on the same simulation model, for example, the texture features of the target object presented on the first sample image and the second sample image are different, the colors presented (such as skin color) are different, the lighting features presented are different, or the special effects presented (such as displaying specific props on the image) are different, etc. The rendering parameters under different image styles can be determined according to the model training requirements or the sample data construction requirements.
[0044] Optionally, the rendering parameters also include background parameters. The background parameters and lighting parameters are randomly adjusted during the rendering process to increase the realism of the rendered image. Moreover, the more times the random adjustments are made, the more types of images are rendered, which helps to improve the diversity of sample data and ensure the model training effect.
[0045] In the disclosed embodiment, for each simulation model, rendering will be performed according to the rendering parameters under the two image styles, and each two images of the target object obtained can be used as a set of paired sample data. Taking the first image style corresponding to a realistic image style and the second image style corresponding to a specific style of an effect image as an example, based on multiple sets of rendering parameters under the first image style, each simulation model is rendered to obtain multiple first sample images with a sense of reality, and based on multiple sets of rendering parameters under the second image style, each simulation model is rendered to obtain multiple second sample images with a specific effect style, such as texture, color, lighting and special effects. At least one feature is different from the first sample image. Specifically, the rendering parameters under the second image style can be determined according to the model training requirements or the sample data construction requirements, and then each two sample images (including a realistic image and an image with a specific effect style) obtained by rendering based on the same simulation model are used as a set of paired sample data for the model training process. It should be understood that the aforementioned examples of the first image style and the second image style are also valid in reverse, that is, the second image style can correspond to a realistic image style, and the first image style can correspond to a specific style of an effect image.
[0046] Figure 3 The schematic diagram of a hand image presenting different image styles provided in the embodiment of the present disclosure can be used to exemplarily illustrate the first sample image and the second sample image of the target object in the embodiment of the present disclosure, but should not be understood as a specific limitation of the embodiment of the present disclosure. Figure 3 As shown, Figure 3 The left image in can be used as an example of a realistic first sample image. Figure 3 The right image in can be used as an example of a second sample image with a specific effect style. For example, the hand in the second sample image has specific texture features. Figure 3 The two hand images in can be used as a set of paired sample data for model training.
[0047] Assume that by adjusting the state parameters of an initial simulation model having the feature representation capability of a target object, N simulation models in different states can be obtained, and then each simulation model is rendered according to rendering parameters under different image styles (assuming that there are M groups of rendering parameters under each image style), then N×M pairs of paired sample data can be obtained. That is, based on the technical solution of the embodiment of the present disclosure, a large number of sample data with different states, textures, colors, lighting, special effects, etc. of the target object can be conveniently constructed, so that the trained model can process any input image and output a corresponding specific effect image.
[0048] In the embodiments of the present disclosure, by utilizing simulation models of the target object in different states and based on different image style requirements, each simulation model is rendered differently to obtain sample images presenting different image styles, which are used as paired sample data for training the image generation model. Since each pair of sample images is rendered based on the same simulation model, the simulation model can accurately characterize the characteristics of the target object, thereby ensuring the image quality of the first sample image and the second sample image. Furthermore, the high-quality sample data ensures the model training effect, and the higher model training effect ensures the personalized image style conversion effect on the target object. Therefore, the embodiments of the present disclosure not only enrich the types of image style conversion, but also ensure the model training effect and the display effect of personalized image style conversion.
[0049] Figure 4 This is a schematic diagram of the structure of an image generating device provided in an embodiment of the present disclosure. The device can be implemented using software and / or hardware and can be integrated on any electronic device with computing capabilities, such as a terminal or a server.
[0050] like Figure 4 As shown, the image generating device 400 provided in the embodiment of the present disclosure may include an initial image acquiring module 401 and a target image determining module 402, wherein:
[0051] An initial image acquisition module 401 is used to acquire an initial image of a target object, where the initial image is an image of a first image style;
[0052] A target image determination module 402 is configured to obtain a target image of a target object based on an output result of the first machine learning model by inputting an initial image into the first machine learning model, wherein the target image is an image of the second image style;
[0053] Wherein, the first machine learning model is obtained based on the second machine learning model; the second machine learning model is obtained by training based on a first sample image of the target object and a second sample image of the target object; the first sample image is an image obtained by rendering a simulation model of the target object in different states based on rendering parameters under a first image style; and the second sample image is an image obtained by rendering a simulation model of the target object in different states based on rendering parameters under a second image style;
[0054] The simulation model is used to construct graphics of the target object in different states, where the different states include different shapes of the target object and / or different postures of the target object.
[0055] Optionally, the first machine learning model is the second machine learning model.
[0056] Optionally, the first machine learning model is trained based on a third sample image of the target object and a fourth sample image of the target object; the third sample image is an image in the style of the first image, and the fourth sample image is an image output by the second machine learning model obtained by inputting the third sample image into the second machine learning model.
[0057] Optionally, the simulation models of the target object in different states are obtained by adjusting state parameters of an initial simulation model of the target object.
[0058] Optionally, the rendering parameters include at least one of the following: texture parameters, color parameters, lighting parameters, and special effect parameters of the target object;
[0059] There is a difference between at least one of the rendering parameters in the first image style and the second image style.
[0060] Optionally, the rendering parameters also include background parameters, and the background parameters and lighting parameters are randomly adjusted during the rendering process.
[0061] Optionally, the target object includes a hand or a foot.
[0062] The image generation device provided in the embodiment of the present disclosure can execute any image generation method provided in the embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method. The contents not described in detail in the embodiment of the device of the present disclosure can refer to the description in any method embodiment of the present disclosure. Figure 5A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure is used to exemplify an electronic device that implements the image generation method provided in an embodiment of the present disclosure. The electronic device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, smart home devices, wearable electronic devices, servers, etc. Figure 5 The electronic device shown is only an example and should not bring any limitation to the functions and occupation scope of the embodiments of the present disclosure.
[0063] like Figure 5 As shown, the electronic device 500 includes one or more processors 501 and a memory 502 .
[0064] The processor 501 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 500 to perform desired functions.
[0065] The memory 502 may include one or more computer program products, and the computer program product may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on a computer-readable storage medium, and the processor 501 may run the program instructions to implement the image generation method provided by the embodiment of the present disclosure, and may also implement other desired functions. Various contents such as input signals, signal components, noise components, etc. may also be stored in the computer-readable storage medium.
[0066] Among them, the image generation method provided by the embodiment of the present disclosure includes: obtaining an initial image of the target object, the initial image is an image of a first image style; by inputting the initial image into a first machine learning model, obtaining a target image of the target object based on the output result of the first machine learning model, and the target image is an image of a second image style; wherein the first machine learning model is obtained based on the second machine learning model; the second machine learning model is trained based on a first sample image of the target object and a second sample image of the target object, the first sample image is an image obtained by rendering a simulation model of the target object in different states based on the rendering parameters under the first image style, and the second sample image is an image obtained by rendering a simulation model of the target object in different states based on the rendering parameters under the second image style; the simulation model is used to construct graphics of the target object in different states, and the different states include different shapes of the target object and / or different postures of the target object.
[0067] It should be understood that the electronic device 500 can also execute other optional implementation schemes provided by the method embodiments of the present disclosure.
[0068] In one example, the electronic device 500 may further include: an input device 503 and an output device 504 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).
[0069] In addition, the input device 503 may also include, for example, a keyboard, a mouse, and the like.
[0070] The output device 504 can output various information to the outside, including the determined distance information, direction information, etc. The output device 504 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.
[0071] Of course, to simplify, Figure 5 Only some of the components related to the present disclosure in the electronic device 500 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 500 may also include any other appropriate components.
[0072] In addition to the above-mentioned methods and devices, the embodiments of the present disclosure also provide a computer program product, which includes a computer program or computer program instructions. When the computer program or computer program instructions are executed by a computing device, the computing device implements any image generation method provided by the embodiments of the present disclosure.
[0073] The computer program product may be written in any combination of one or more programming languages to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user electronic device, partially on the user electronic device, as a separate software package, partially on the user electronic device and partially on a remote electronic device, or entirely on a remote electronic device.
[0074] In addition, the embodiments of the present disclosure may also provide a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a computing device, the computing device implements any image generation method provided by the embodiments of the present disclosure.
[0075] Among them, the image generation method provided by the embodiment of the present disclosure includes: obtaining an initial image of the target object, the initial image is an image of a first image style; by inputting the initial image into a first machine learning model, obtaining a target image of the target object based on the output result of the first machine learning model, and the target image is an image of a second image style; wherein the first machine learning model is obtained based on the second machine learning model; the second machine learning model is trained based on a first sample image of the target object and a second sample image of the target object, the first sample image is an image obtained by rendering a simulation model of the target object in different states based on the rendering parameters under the first image style, and the second sample image is an image obtained by rendering a simulation model of the target object in different states based on the rendering parameters under the second image style; the simulation model is used to construct graphics of the target object in different states, and the different states include different shapes of the target object and / or different postures of the target object.
[0076] It should be understood that when the computer program instructions are executed by a computing device, they can also enable the computing device to implement other optional implementation schemes provided by the method embodiments of the present disclosure.
[0077] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
[0078] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0079] The above are only specific embodiments of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to these embodiments herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.
Claims
1. An image generation method, characterized in that: include: Acquire an initial image of the target object, where the initial image is an image of a first image style; By inputting the initial image into a first machine learning model, a target image of the target object is obtained based on an output result of the first machine learning model, wherein the target image is an image of the second image style; Wherein, the first machine learning model is obtained based on the second machine learning model; the second machine learning model is obtained by training based on the first sample image of the target object and the second sample image of the target object; the first sample image is an image obtained by rendering the simulation model of the target object in different states based on the rendering parameters under the first image style; and the second sample image is an image obtained by rendering the simulation model of the target object in different states based on the rendering parameters under the second image style; The simulation model is used to construct graphics of the target object in different states, wherein the different states include different shapes of the target object and / or different postures of the target object; The first machine learning model is obtained based on the second machine learning model, including: The first machine learning model is trained based on a third sample image of the target object and a fourth sample image of the target object; the third sample image is an image of the style of the first image, and the fourth sample image is an image output by the second machine learning model obtained by inputting the third sample image into the second machine learning model.
2. The method according to claim 1, characterized in that The first machine learning model is obtained based on the second machine learning model, and also includes: the first machine learning model is the second machine learning model.
3. The method according to claim 1, characterized in that The simulation models of the target object in different states are obtained by adjusting the state parameters of the initial simulation model of the target object.
4. The method according to claim 1, characterized in that The rendering parameters include at least one of the following: texture parameters, color parameters, lighting parameters and special effect parameters of the target object; There is a difference between at least one of the rendering parameters in the first image style and the second image style.
5. The method according to claim 4, characterized in that The rendering parameters also include background parameters, and the background parameters and the illumination parameters are randomly adjusted during the rendering process.
6. The method according to claim 1, characterized in that The target object includes a hand or a foot.
7. An image generating device, characterized in that: include: An initial image acquisition module, used to acquire an initial image of the target object, wherein the initial image is an image of a first image style; a target image determination module, configured to obtain a target image of the target object based on an output result of the first machine learning model by inputting the initial image into a first machine learning model, wherein the target image is an image of a second image style; Wherein, the first machine learning model is obtained based on the second machine learning model; the second machine learning model is obtained by training based on the first sample image of the target object and the second sample image of the target object; the first sample image is an image obtained by rendering the simulation model of the target object in different states based on the rendering parameters under the first image style; and the second sample image is an image obtained by rendering the simulation model of the target object in different states based on the rendering parameters under the second image style; The simulation model is used to construct graphics of the target object in different states, wherein the different states include different shapes of the target object and / or different postures of the target object; The first machine learning model is obtained based on the second machine learning model, including: The first machine learning model is trained based on a third sample image of the target object and a fourth sample image of the target object; the third sample image is an image of the style of the first image, and the fourth sample image is an image output by the second machine learning model obtained by inputting the third sample image into the second machine learning model.
8. An electronic device, characterized in that: The electronic device comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the electronic device implements the image generation method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a computing device, the computing device implements the image generation method according to any one of claims 1 to 6.
Citation Information
Patent Citations
Image Style Transfer for Three-Dimensional Models
CN110084874A
Style migration model training method and device and style migration method and device
CN113610989A
Stylized image generation method and device, electronic equipment and storage medium
CN114419300A
Learning painting styles for painterly rendering
US20150310305A1
Augmented reality method and system for design
US20190164340A1
Cited By
Generating image method and apparatus, device, and medium
US12633014B2
