Image generation method and electronic device

Abstract

The present application relates to the field of image processing technology, and provides an image generation method and electronic device. The method constructs an image generation parameter set based on a real image set, wherein the image generation parameter set includes multiple groups of generation parameters, and each group of generation parameters corresponds to a real image in the real image set; generates a corresponding first image based on each group of generation parameters; determines the authenticity of the first image, and selects the first image corresponding to the authenticity greater than a preset threshold as the second image; if the number of the second images is less than the preset number, updates the image generation parameter set based on a genetic algorithm to obtain an updated image generation parameter set; and generates a target image based on the updated image generation parameter set. The above method can generate a high-quality target image based on rendering technology.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular, to an image generating method and an electronic device. Background Art

[0002] With the rise of deep learning technology, the demand for training data in areas such as image recognition has gradually increased. However, in some areas that require recognition of specific objects, such as human posture recognition and motion analysis, the high cost of collecting a large amount of real data to build a training data set and the potential for privacy protection issues make it difficult to obtain high-quality training data sets with a large amount of data, which in turn leads to poor model training results. Summary of the invention

[0003] The present application discloses an image generation method and an electronic device, which can solve the technical problem in the related art that it is difficult to obtain a high-quality training data set with a large amount of data, and the resulting poor model training effect.

[0004] An embodiment of the present application provides an image generation method, which includes: constructing an image generation parameter set based on a real image set, the image generation parameter set including multiple groups of generation parameters, each group of generation parameters corresponding to a real image in the real image set; generating a corresponding first image based on each group of generation parameters; determining the authenticity of the first image, and selecting the first image corresponding to the authenticity greater than a preset threshold as the second image; if the number of the second images is less than the preset number, updating the image generation parameter set based on a genetic algorithm to obtain an updated image generation parameter set; generating a target image based on the updated image generation parameter set.

[0005] In one embodiment, the method further comprises: determining the preset number based on a preset percentage and the number of the plurality of groups of generation parameters.

[0006] In one embodiment, the method further includes: if the number of the second images is greater than or equal to the preset number, using the second images as the target images.

[0007] In one embodiment, each set of updated generation parameters in the updated image generation parameter set corresponds to a target image, and the method further includes: using the multiple target images to perform downstream model training tasks corresponding to the real image set.

[0008] An embodiment of the present application provides an image generating device, which includes: a construction module, which is used to construct an image generation parameter set based on a real image set, and the image generation parameter set includes multiple groups of generation parameters, and each group of generation parameters corresponds to a real image in the real image set; a generation module, which is used to generate a corresponding first image based on each group of generation parameters; a determination module, which is used to determine the authenticity of the first image, and select the first image corresponding to the authenticity greater than a preset threshold as the second image; an update module, which is used to update the image generation parameter set based on a genetic algorithm to obtain an updated image generation parameter set if the number of the second images is less than a preset number; the generation module is also used to generate a target image based on the updated image generation parameter set.

[0009] An embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the processor is configured to implement the image generating method when executing a computer program stored in the memory.

[0010] An embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the image generating method is implemented.

[0011] The image generation method described in the present application constructs an image generation parameter set based on a real image set, wherein the image generation parameter set includes multiple groups of generation parameters, each group of generation parameters corresponds to a real image in the real image set; generates a corresponding first image based on each group of generation parameters; determines the authenticity of the first image, and selects the first image corresponding to the authenticity greater than a preset threshold as the second image; if the number of the second images is less than the preset number, updates the image generation parameter set based on a genetic algorithm to obtain an updated image generation parameter set; and generates a target image based on the updated image generation parameter set. The real image set and the stable diffusion model can be combined to efficiently generate high-quality and diverse target images by constructing an image generation parameter set and optimizing it using a genetic algorithm, which can effectively solve the problems of scarce training data, insufficient authenticity and diversity, and inconsistent data distribution in fields such as human posture recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a structural diagram of an electronic device provided in one embodiment of the present application.

[0013] Figure 2 It is a flowchart of an image generation method provided by an embodiment of the present application.

[0014] Figure 3 This is an example diagram of a rendered image provided by an embodiment of the present application.

[0015] Figure 4This is an example diagram of a character image provided by an embodiment of the present application.

[0016] Figure 5 This is an example diagram of a background image provided by an embodiment of the present application.

[0017] Figure 6 This is a flowchart of a method for updating an image generation parameter set provided by an embodiment of the present application.

[0018] Figure 7 This is a flowchart of a method for updating selected parent parameters provided by an embodiment of the present application.

[0019] Figure 8 It is a flowchart of an image generation method provided by another embodiment of the present application.

[0020] Fig. 9 This is a flowchart of a method for determining the authenticity of a first image provided by an embodiment of the present application.

[0021] Fig.10 It is a structural diagram of an image generating device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0023] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein in the specification of this application are only for the purpose of describing the embodiments in one embodiment and are not intended to limit this application.

[0024] It should be noted that in this application, "at least one" means one or more, and "more than one" means two or more than two. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claims and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or sequence.

[0025] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a concrete way. The following embodiments and features in the embodiments may be combined with each other without conflict.

[0026] In one embodiment, with the rapid development of artificial intelligence technology, deep learning, as one of its core driving forces, has shown unprecedented potential in multiple fields such as image recognition and natural language processing. Especially in the field of image recognition, deep learning models can achieve efficient learning and recognition of complex visual features by feeding large-scale training data, which has greatly promoted the innovation of technologies such as face recognition, autonomous driving, and medical imaging diagnosis. However, in specific application scenarios such as human posture recognition and motion analysis, the training of deep learning models faces a series of challenges, which mainly come from the difficulty of obtaining and processing high-quality training data.

[0027] Human posture recognition and motion analysis require the model to accurately capture and understand the subtle movements and posture changes of the human body, which places extremely high demands on the precision and diversity of training data. Traditional data collection methods, such as manual shooting and labeling, are not only time-consuming and labor-intensive, but also costly, and may also raise ethical and legal issues in scenarios involving personal privacy. In addition, even the collected data may have uneven data quality due to factors such as shooting angle, lighting conditions, and human occlusion, making it difficult to meet the needs of deep learning models for high-quality, large-scale data sets.

[0028] In order to alleviate the problem of data scarcity, relevant technologies have explored the use of computer graphics, artificial intelligence generation technology (such as generative adversarial networks) and other means to automatically generate annotated human data. Although these technologies can generate seemingly real human images to a certain extent, they still have significant deficiencies in ensuring the authenticity and diversity of data. The generated images often lack complex backgrounds, lighting changes and other content in the real world, which poses a challenge to the generalization ability of the model.

[0029] More importantly, even if the image data is generated by the above technology, it is difficult to ensure that its distribution is consistent with the real-world data set. This distribution difference may cause the performance of the model in real application scenarios to be greatly reduced, which is the so-called "domain transfer" problem. No matter how well the model is trained on synthetic data, once it faces the complex and changeable situations in the real world, it may make recognition errors or performance degradation.

[0030] Based on the above, in the training of deep learning models in specific fields such as human posture recognition and motion analysis, there are problems such as scarcity of high-quality training data, insufficient authenticity and diversity of automatically generated data, and inconsistent distribution between generated data and real data sets. These problems not only limit the improvement of model performance, but also hinder the effective application of related technologies in more practical scenarios.

[0031] To solve the above problems, an embodiment of the present application provides an image generation method, which can combine a real image set with a stable diffusion model, construct an image generation parameter set and use genetic algorithm optimization to efficiently generate high-quality and diverse target images, and can effectively solve the problems of scarce training data, insufficient authenticity and diversity, and inconsistent data distribution in fields such as human posture recognition.

[0032] Figure 1 The electronic device 10 can be a computer, a server, a mobile phone, a tablet computer, a notebook computer, or other electronic devices. The specific type of the electronic device is not limited in any way in the embodiment of the present application.

[0033] like Figure 1 As shown, the electronic device 10 may include a communication module 101, a memory 102, a processor 103, an input / output (I / O) interface 104 and a bus 105. The processor 103 is coupled to the communication module 101, the memory 102 and the I / O interface 104 through the bus 105.

[0034] The communication module 101 may include a wired communication module and / or a wireless communication module. The wired communication module may provide one or more wired communication solutions such as universal serial bus (USB), controller area network bus (CAN), etc. The wireless communication module may provide one or more wireless communication solutions such as wireless fidelity (Wi-Fi), bluetooth (BT), mobile communication network, frequency modulation (FM), near field communication technology (NFC), infrared technology (IR), etc.

[0035] The memory 102 may include one or more random access memories (RAM) and one or more non-volatile memories (NVM). The random access memory can be directly read and written by the processor 103, and can be used to store executable programs (such as machine instructions) of the operating system or other running programs, and can also be used to store user and application data. The random access memory may include static random-access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), etc.

[0036] The non-volatile memory may also store executable programs and user and application data, etc., and may be loaded into the random access memory in advance for direct reading and writing by the processor 103. The non-volatile memory may include a disk storage device and a flash memory.

[0037] The memory 102 is used to store one or more computer programs. The one or more computer programs are configured to be executed by the processor 103. The one or more computer programs include a plurality of instructions, and when the plurality of instructions are executed by the processor 103, the image generation method executed on the electronic device 10 can be implemented.

[0038] In other embodiments, the electronic device 10 further includes an external memory interface for connecting to an external memory to expand the storage capacity of the electronic device 10 .

[0039] The processor 103 may include one or more processing units, for example, the processor 103 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0040] The processor 103 provides computing and control capabilities. For example, the processor 103 is used to execute a computer program stored in the memory 102 to implement the above-mentioned image generation method.

[0041] The I / O interface 104 is used to provide a channel for user input or output. For example, the I / O interface 104 can be used to connect various input and output devices, such as a mouse, keyboard, touch device, display screen, etc., so that the user can enter information or visualize information.

[0042] The bus 105 is at least used to provide a channel for mutual communication among the communication module 101 , the memory 102 , the processor 103 , and the I / O interface 104 in the electronic device 10 .

[0043] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 10. In other embodiments of the present application, the electronic device 10 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0044] Figure 2 is a flow chart of an image generation method provided by an embodiment of the present application. The image generation method is applied to an electronic device, such as Figure 1 The electronic device 10 in the flowchart specifically includes the following steps. According to different requirements, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0045] Step S21, constructing an image generation parameter set based on a real image set.

[0046] In one embodiment, in order to generate a large amount of high-quality image training data, a real image set consisting of a small number of high-quality real images can be obtained. In one example, real images can be obtained by web crawlers, public data sets, or self-photographing, accepting user input, etc., and the real images are screened and preprocessed to construct a real image set. For example, preprocessing can include but is not limited to image denoising processing, size adjustment processing, etc.

[0047] In one example, the real images in the real image set may correspond to the same scene, for example, the backgrounds in different real images belong to the same scene, so image training data for the same scene can be generated based on the real image set. In another example, the real images in the real image set may include different scenes, character postures, and backgrounds, so image training data with diversity and comprehensiveness can be generated based on the real image set. The present application does not impose any specific restrictions on the real images in the real image set, and real images can be acquired and a real image set can be constructed based on actual needs.

[0048] In one embodiment, a set of generation parameters can be obtained based on each real image in the real image set, and multiple sets of generation parameters can be obtained based on all real images, and the image generation parameter set can be obtained based on the multiple sets of generation parameters. For example, the image generation parameter set includes multiple sets of generation parameters, and each set of generation parameters corresponds to a real image in the real image set.

[0049] In one embodiment, in order to obtain a generated image, the background and the characters in the image may be generated separately. Therefore, each set of generation parameters may include character generation parameters for generating characters, and background generation parameters for generating background.

[0050] In one embodiment, the character generation parameters include rendering parameters and optimization parameters, wherein the rendering parameters are used to render and obtain a rendered image of the character, and the optimization parameters are used to optimize the rendered image and obtain a character image.

[0051] In one embodiment, rendering parameters can be used to control the rendering process of the character, and the rendering parameters may include but are not limited to one or more of the following parameters: camera intrinsic parameters, camera external parameters relative to the ground, the position of the character on the ground, etc. When obtaining the rendering parameters, the randomly generated parameters can be used as rendering parameters through a random generation method. In one example, each group of rendering parameters includes multiple parameters, and the parameter types corresponding to each group of rendering parameters are the same, and different groups of rendering parameters include at least one different parameter. For example, the first group of rendering parameters includes {camera intrinsic parameter n1, camera external parameter w1 relative to the ground, character position z1 on the ground}, the second group of rendering parameters includes {camera intrinsic parameter n1, camera external parameter w2 relative to the ground, character position z2 on the ground}, and the third group of rendering parameters includes {camera intrinsic parameter n3, camera external parameter w1 relative to the ground, character position z2 on the ground}.

[0052] In one embodiment, the optimization parameters include character description words determined based on the real image, and the optimization parameters are used to optimize the details of the rendered image, for example, adjusting the expression and posture of the character to make it more consistent with the description. When obtaining the character description words, a real image can be randomly selected from the real image set, and the character description words in the selected real image can be obtained through the multimodal large model, and the corresponding character generation parameters are determined according to the character description words.

[0053] In one example, the multimodal large model can be a trained and well-performing multimodal large model that can process image data and generate text description information corresponding to the image data. A randomly selected real image is input into the multimodal large model, and the model parses the real image and extracts descriptive information related to the person. Descriptive words related to the person are extracted from the descriptive information related to the person output by the model, and the descriptive words can accurately reflect information such as the characteristics and posture of the person in the image. For example, for a real image of a person standing in a shopping mall, the person description words can be "person, standing".

[0054] In another example, character description words may be mapped to specific character generation parameters, for example, by using preset mapping rules or machine learning models, so that the character description words can be converted into quantifiable numerical values ​​or other patterns.

[0055] In one embodiment, the background generation parameters are used to generate the background. When obtaining the background generation parameters, the background descriptor in the real image can be obtained through the multimodal large model, and the corresponding background generation parameters can be determined according to the background descriptor. The method of obtaining the background descriptor is similar to the method of obtaining the person descriptor, and the description is not repeated. For example, for a real image of a person standing in a shopping mall, the background descriptor can be "shopping mall".

[0056] In one embodiment, the character generation parameters and the background generation parameters can be spliced ​​into a vector form in a preset order to obtain a set of generation parameters. The order can be set according to actual needs, and the present application does not impose specific restrictions on this. In one example, each set of generation parameters includes multiple parameters, the parameter types corresponding to each set of generation parameters are the same, the arrangement order of the parameter types in each set of generation parameters is the same, and different sets of generation parameters include at least one different parameter. For example, the first set of generation parameters includes {camera intrinsic parameter n1, camera external parameter w1 relative to the ground, character position z1 on the ground, "person, standing", "shopping mall"}, the second set of generation parameters includes {camera intrinsic parameter n1, camera external parameter w2 relative to the ground, character position z2 on the ground, "person, standing", "shopping mall"}, and the third set of rendering parameters includes {camera intrinsic parameter n3, camera external parameter w1 relative to the ground, character position z2 on the ground, "person, walking", "beach"}.

[0057] In another embodiment, the image generation parameter set also includes other parameters, such as the gender, age, skin color, hairstyle, clothing, posture, and viewing angle parameters of the person, the internal and external parameters of the camera corresponding to the background image, etc. These other parameters may not belong to each of the above-mentioned groups of generation parameters, and can be used to increase the diversity of data by random sampling.

[0058] Based on the above embodiment, by constructing an image set consisting of high-quality real images and extracting generation parameters based on each real image, an image generation parameter set containing multiple groups of generation parameters can be obtained. These parameters not only cover the generation of characters and backgrounds, but also include details such as rendering parameters and optimization parameters, which can help to make the generated images more consistent with the real description in the subsequent image generation process. In addition, by introducing other randomly sampled parameters, such as the gender and age of the characters, the diversity and comprehensiveness of the generated images are further increased, providing effective support for generating a large amount of high-quality image training data.

[0059] Step S22: Generate a corresponding first image based on each set of generation parameters.

[0060] In one embodiment, when generating the corresponding first image based on each set of generation parameters, a character image and a background image may be generated separately, and the character image and the background image may be fused to obtain the first image.

[0061] In one embodiment, when generating a character image, the character image can be generated based on the character generation parameters and the corresponding real image. The character rendering image can be generated based on the rendering parameters, and the rendering image can be optimized based on the optimization parameters and the corresponding real image to obtain the character image.

[0062] In one embodiment, a preset rendering engine and tool may be used to generate a rendered image of a character according to the above rendering parameters. For example, the rendering engine and tool may include but are not limited to Cocos2d-x, Unity, etc. In other embodiments, a three-dimensional model of each character may be created based on a preset modeling software (such as Maya, Blender, 3ds Max, etc.), and a rendered image of each character may be obtained based on the three-dimensional model.

[0063] refer to Figure 3 The figure is an example of a rendered image provided by an embodiment of the present application. Figure 3 Figure (a) in FIG. 1 represents a rendered image. In one embodiment, a parameter image corresponding to the rendered image may also be generated, for example: Figure 3 The depth image shown in (b) is Figure 3 The mask image shown in (c) of Figure 3 The action image shown in (d) in Figure 1. The lines in the action image represent the connections between the key points of the skeleton.

[0064] In one embodiment, the rendered image, the optimization parameters, and the corresponding real image may be input into a preset stable diffusion (SD) model, and the SD model may be used to optimize the character in the rendered image based on the optimization parameters and the real image, so that the style features of the character in the first image obtained are consistent with the style features of the character in the real image. For example, the SD model may include but is not limited to models such as an open source Image Prompt Adapter, and the style features may include but are not limited to features such as texture, edge, and color.

[0065] refer to Figure 4 As shown, it is an example diagram of a character image provided by an embodiment of the present application, combined with Figure 3 From the rendered image shown in Figure (a), it can be seen that the character image has been significantly optimized compared to the rendered image and is closer to the character in the real image.

[0066] In one example, during the optimization process of the SD model, the character description words in the optimization parameters can be used as prompt words to guide the SD model to perform customized processing on the rendered image. The prompt words are like instructions and can be used to guide the SD model to perform corresponding image processing tasks.

[0067] In one example, the stable diffusion model can extract the character style features of the real image, and use the character style features of the real image and the character description words as the first control information. The stable diffusion model can optimize the character in the rendered image according to the first control information. In one example, the optimization process may include multiple iterations, and each iteration will be adjusted according to the current results and optimization goals. After optimization, the character style features in the first image should be consistent with the style features of the character in the real image. The optimization results can be evaluated and adjusted to ensure that they meet actual needs.

[0068] In one embodiment, since the character image is generated based on the character description words obtained from the real image, an association relationship can be established between the character image and the corresponding real image, wherein one real image can have an association relationship with multiple character images.

[0069] In one embodiment, when generating a background image, the background image can be generated based on the background generation parameters and the corresponding real image. In one example, the background generation parameters and the real image can be input into the SD model, and the background style features of the real image can be extracted using the SD model. The background style features of the real image and the background description words in the background generation parameters are used as the second control information. The SD model can optimize the background in the rendered image according to the second control information, which is similar to the method for generating a character image, and will not be described again.

[0070] refer to Figure 5 FIG. 1 is an example of a background image provided by an embodiment of the present application, wherein a plurality of corresponding background images are obtained according to the real image and the background description words "blue sky, white clouds, seagulls, beach, sea" corresponding to the real image.

[0071] In one embodiment, since the background image is generated based on the background description words obtained from the real image, an association relationship can be established between the background image and the corresponding real image, wherein one real image can have an association relationship with multiple background images.

[0072] In one embodiment, a character image is fused with a background image to obtain a first image. For example, based on image fusion technology, a character image associated with the same real image can be fused with a background image to obtain a first image. In one example, a character image can be fused with multiple background images to obtain multiple first images. In another example, multiple character images can be fused with the same background image to obtain a first image. The present application does not specifically limit the method of fusing a character image with a background image to obtain a first image.

[0073] Based on the above embodiments, by using a stable diffusion model, combined with a preset rendering engine and tools and optimization parameters, character images and background images that are consistent with the style of real images can be generated respectively. During the generation process, the stable diffusion model not only significantly enhances the details in the image, making the texture, edges and other features of the character and background more vivid and realistic, but also effectively suppresses image noise, making the overall picture smoother and clearer. By fusing the optimized character image with the background image, a first image that is highly consistent with the style of the real image can be obtained, and the feature distribution in the first image is consistent with the distribution of the real image.

[0074] Step S23, determining the authenticity of the first image, and selecting the first image corresponding to the authenticity greater than a preset threshold as the second image.

[0075] In one embodiment, since the character image and the background image in the first image are generated separately, the first image is likely to be an unreasonable image. For example, the character standing in the first image is not on the ground in the background, but on the sea surface, which is obviously not in line with the actual situation. Or, the standing positions of multiple characters in the first image overlap, which is obviously unreasonable. Therefore, the first image may not be directly used as training data for downstream training tasks.

[0076] In one embodiment, in order to determine whether the first image can be used as training data, the authenticity of the first image can be determined by a pre-trained discriminant model, and the authenticity is used to indicate the probability that the first image is a real image. The first image corresponding to the authenticity greater than a preset threshold can be selected as the second image, and the second image can be regarded as a selected high-quality image close to the real image. Therefore, the second image can be used as training data for downstream training tasks. Among them, the preset threshold can be set according to actual needs, such as 80%, and this application does not impose specific restrictions on this.

[0077] In one embodiment, in order to enable the trained discriminant model to accurately evaluate the authenticity of the first image, especially the authenticity of the image of the person, a preprocessing step can be used to minimize the interference of the background image in the training data of the discriminant model when training the discriminant model.

[0078] In one embodiment, preprocessing includes but is not limited to: obtaining a fused image based on the first image and the corresponding real image. For example, a person cutout may be performed on the real image according to a preset preprocessing probability, and the person cutout is fused to the background image corresponding to the first image to obtain a fused image. The preprocessing probability may be set according to actual needs, such as 50%, and this application does not impose specific restrictions on this. For example, for a real image, the real image may be preprocessed with a probability of 50% to obtain a fused image, and the real image may not be preprocessed with a probability of 50%.

[0079] In one embodiment, the authenticity label of the real image can be set to 1, indicating that the authenticity of the person image in the real image is real. In addition, since the person in the fused image is obtained by cutting out the person in the real image, the authenticity of the person image in the fused image is also real, and therefore, the authenticity label of the fused image can also be set to 1. In this way, although the background image in the fused image is the background generated according to the generation parameters, since the authenticity label of the fused image is 1, when training the discriminant model, the model can be not disturbed by the background image and focus on learning the authenticity of the person.

[0080] In one embodiment, supervised training may be performed on a preset neural network model based on the real image, the authenticity label corresponding to the real image, the fused image, and the authenticity label of the fused image to obtain a discriminant model.

[0081] In one example, the preset neural network model can be a discriminator in a generative adversarial network. When training the discriminator, real images and fused images with a authenticity label of 1 can be used as true samples, and random noise images can be used as false samples with a authenticity label of 0, and the discriminator can be trained based on true samples and false samples. During the training process, true samples and false samples can be input into the discriminator respectively, and the discriminator is used to predict the authenticity of each sample. The loss of the discriminator is determined using a preset loss function (such as cross entropy loss), and the discriminator training parameters (such as learning rate, batch size, network architecture, etc.) are updated through the back propagation algorithm to minimize the loss function until the loss converges or the training reaches the number of iterations to obtain a discriminant model.

[0082] In one example, regularization techniques (such as L2 regularization, Dropout, etc.) can also be introduced in the above training process to prevent overfitting. You can try to use different neural network architectures (such as convolutional neural networks, recurrent neural networks, etc.) to build a discriminant model to find the model that best suits the current task.

[0083] Based on the above embodiment, the character in the real image can be fused into the generated background through the character cutout technology to form a fused image, and its authenticity label is set to 1. Subsequently, the fused image and the real image are used to perform supervised training on the discriminant model. Regularization technology is introduced during the training process to prevent overfitting, and different neural network architectures are tried. The trained discriminant model can focus on learning the authenticity of the character without being disturbed by the background image, thereby improving the prediction accuracy and generalization ability of the model.

[0084] Step S24: if the number of the second images is less than the preset number, the image generation parameter set is updated based on the genetic algorithm to obtain an updated image generation parameter set.

[0085] In one embodiment, if the number of second images is less than a preset number, and the number of training data sets constructed based on the second images is insufficient, the image generation parameter set can be updated and expanded based on the genetic algorithm according to the image generation parameters corresponding to the second images to obtain an updated image generation parameter set. The preset number can be set according to actual needs, and this application does not impose specific restrictions on this. In one example, the preset number can be determined based on a preset percentage and the number of multiple sets of generation parameters. For example, the preset percentage can be 50%, and this application does not impose specific restrictions on this.

[0086] In one embodiment, when updating the image generation parameter set based on the genetic algorithm, multiple groups of generation parameters corresponding to the second image can be screened out from the multiple groups of generation parameters, and these parameters can be regarded as multiple groups of parent parameters. Weights are assigned to the corresponding group of parent parameters according to the authenticity of each second image. The weight assignment can be based on the authenticity, and the higher the authenticity, the greater the weight. These weights can be used to selectively select from the multiple groups of parent parameters, and the selected parent parameters can be updated based on the principle of the genetic algorithm.

[0087] In one example, a genetic algorithm can iteratively optimize a parameter set by simulating natural selection and genetic mechanisms (such as crossover, mutation, etc.). The parent parameters updated by the genetic algorithm can be regarded as updated generation parameters, and these updated generation parameters can be used to construct a new image generation parameter set. In one example, a method for updating an image generation parameter set based on a genetic algorithm can be referred to Figure 6 Flowchart shown.

[0088] Based on the above embodiment, the image generation parameter set can be updated and expanded by a genetic algorithm to obtain an updated image generation parameter set. In this way, a number of generated images greater than or equal to a preset number can be obtained based on the updated image generation parameter set.

[0089] In another embodiment, if the number of the second images is greater than or equal to a preset number, the second images may be used as target images without updating the image generation parameter set.

[0090] Step S25, generating a target image based on the updated image generation parameter set.

[0091] In one embodiment, a target image can be generated according to each set of generation parameters in the updated image generation parameter set. In this way, a sufficient number of target images can be generated based on the updated image generation parameter set, and multiple target images can be used to perform downstream model training tasks corresponding to the real image set. For example, downstream model training tasks include but are not limited to training tasks of models such as face recognition, human posture recognition, and person tracking. In this way, based on the above method, the difficulty of obtaining training data can be reduced, the amount of training data can be enriched, and the training effect of the model can be improved.

[0092] The image generation method provided in the embodiment of the present application can combine a real image set with a stable diffusion model, and by constructing an image generation parameter set and optimizing it using a genetic algorithm, it can efficiently generate high-quality and diverse target images, and can effectively solve the problems of scarce training data, insufficient authenticity and diversity, and inconsistent data distribution in fields such as human posture recognition.

[0093] In one embodiment, Figure 6 This is a flow chart of a method for updating an image generation parameter set based on a genetic algorithm provided in an embodiment of the present application. The method is applied in an electronic device and specifically includes the following steps. According to different requirements, the order of the steps in the flow chart can be changed, and some can be omitted.

[0094] Step S61, selecting multiple groups of generation parameters corresponding to the second image from the multiple groups of generation parameters, using the selected multiple groups of generation parameters as multiple groups of parent parameters, and determining the weight of a corresponding group of parent parameters according to the authenticity of each second image.

[0095] In one embodiment, the second image is a high-quality image obtained through authenticity screening, so the multiple sets of generation parameters corresponding to the second image can be regarded as multiple sets of parent parameters as the starting point for subsequent genetic algorithm optimization. Among them, weights can also be assigned to the corresponding set of parent parameters according to the authenticity of each second image. The higher the authenticity, the closer the second image generated by the set of generation parameters is to the real image, so the weight of this set of parameters should also be greater.

[0096] In one embodiment, the weight of each group of parameters can be set to be proportional to the truth, and the set weights can be normalized. After normalization, the sum of the weights of all parent parameters is 1, which can ensure that in the subsequent sampling process, the weights can correctly reflect the importance of each group of parameters.

[0097] Step S62: selecting parent parameters from multiple groups of parent parameters according to weights, and updating the selected parent parameters based on a genetic algorithm.

[0098] In one embodiment, the parent parameters can be sampled according to the weights, that is, one or more sets of parent parameters are randomly selected as candidates for the next generation according to the weight distribution. During the sampling process, the parent parameters with larger weights have a higher probability of being selected, which is in line with the principle of "survival of the fittest" in natural selection.

[0099] In one embodiment, the selected parent parameters can be subjected to subsequent genetic algorithm operations to generate new generation parameters. In one example, the genetic algorithm operations include but are not limited to crossover and mutation. The crossover operation can combine the beneficial features of different parent parameters to generate new parameter combinations. The mutation operation can make small random adjustments to the parameters to increase the diversity of the parameters. In one example, the method for updating the selected parent parameters based on the genetic algorithm can refer to Figure 7 Flowchart shown.

[0100] Step S63: construct an updated image generation parameter set according to the updated parent parameters.

[0101] In one embodiment, a new image generation parameter set may be constructed using the updated parent parameters. If the number of updated parent parameters is less than a preset number, parent parameters are continuously selected from multiple sets of parent parameters according to weights, and the continuously selected parent parameters are updated until the number of updated parent parameters obtained is greater than or equal to the preset number.

[0102] Based on the above embodiment, the genetic algorithm can be used to effectively update and expand the image generation parameter set. By selecting the generation parameters corresponding to the high-quality second image as the parent, and assigning weights to them according to the authenticity of the second image, it is ensured that the weights can accurately reflect the importance of each group of parameters. The parent parameters are sampled according to these weights, and operations such as crossover and mutation in the genetic algorithm are applied to generate new parameters, thereby constructing an updated image generation parameter set containing a sufficient number of groups of generation parameters, which can provide strong support for generating a sufficient number of target images.

[0103] In one embodiment, Figure 7 This is a flow chart of a method for updating selected parent parameters based on a genetic algorithm provided in an embodiment of the present application. The method is applied in an electronic device and specifically includes the following steps. According to different requirements, the order of the steps in the flow chart can be changed, and some can be omitted.

[0104] Step S71, randomly select two groups of parent parameters from multiple groups of parent parameters based on weights, cross-replace the parameters in the two selected groups of parent parameters according to a preset crossover probability, and obtain two groups of child parameters, and use each group of child parameters as a group of updated parent parameters.

[0105] In one example, if there are 5 groups of parent parameters, the weight of the first group of parent parameters is 0.2, the weight of the second group of parent parameters is 0.3, the weight of the third group of parent parameters is 0.05, the weight of the fourth group of parent parameters is 0.1, and the weight of the fifth group of parent parameters is 0.35. When two groups of parent parameters are randomly selected from multiple groups of parent parameters according to the weights, since the weights of the second group of parent parameters and the fifth group of parent parameters are the two largest, the second group of parent parameters and the fifth group of parent parameters can be selected.

[0106] In one embodiment, when cross-replacing parameters in two selected sets of parent parameters according to a preset cross-over probability, since the parameter types in each set of parent parameters are the same and the arrangement order of the parameter types is the same, parameters of the parameter types in the same position can be selected for cross-replacing according to the cross-over probability. Among them, the cross-over probability can be set according to actual needs, for example, 50%, and the present application does not impose specific restrictions on this. For example, if the cross-over probability is 50%, there is a 50% probability of performing cross-replacing on the parameters in the two selected sets of parent parameters, and there is also a 50% probability of not performing cross-replacing on the parameters in the two selected sets of parent parameters.

[0107] In one example, the second group of parent parameters includes {camera intrinsic parameter n3, camera external parameter w1 relative to the ground, person's position on the ground z2, "person, walking", "beach"}, and the fifth group of parent parameters includes {camera intrinsic parameter n3, camera external parameter w2 relative to the ground, person's position on the ground z2, "person, lying down", "grass"}.

[0108] In one example, the camera intrinsic parameters in the second group of parent parameters and the fifth group of parent parameters are cross-replaced according to the crossover probability, and two groups of child parameters can be obtained, including: a first group of child parameters {camera intrinsic parameter n3, camera external parameter w1 relative to the ground, person's position on the ground z2, "person, walking", "beach"}, a second group of child parameters {camera intrinsic parameter n3, camera external parameter w2 relative to the ground, person's position on the ground z2, "person, lying down", "grass"}.

[0109] In another example, the character description words in the second group of parent parameters and the fifth group of parent parameters are cross-replaced according to the crossover probability, and two groups of child parameters can be obtained, including: the third group of child parameters {camera intrinsic parameter n3, camera external parameter w1 relative to the ground, character position z2 on the ground, "person, lying down", "beach"}, the fourth group of child parameters {camera intrinsic parameter n3, camera external parameter w2 relative to the ground, character position z2 on the ground, "person, walking", "grass"}.

[0110] Based on the above embodiment, by randomly selecting two groups of parent parameters based on weights, and performing crossover replacement on the same-rank parameter types of the selected parent parameters according to a preset crossover probability, an updated parent parameter group can be obtained. This not only makes full use of the diversity of the parent parameters, but also introduces new parameter combinations through the crossover operation, thereby effectively improving the diversity of the parameters.

[0111] Step S72, randomly updating the parameters in each group of offspring parameters according to a preset disturbance probability, obtaining the mutated parameters corresponding to each group of offspring parameters, and using each group of mutated parameters as a group of updated parent parameters.

[0112] In one embodiment, when randomly updating the parameters in each group of offspring parameters according to a preset disturbance probability, random disturbance can be added to at least one parameter in each group of offspring parameters. For example, for parameters of numerical type, a certain percentage of the numerical value can be increased or decreased on the basis of the original parameter; for parameters of text type, other words can be randomly selected from a preset vocabulary for replacement. Among them, the disturbance probability can be set according to actual needs, for example, 49%, and this application does not impose specific restrictions on this. For example, if the disturbance probability is 49%, there is a 49% probability of randomly updating a parameter in a group of offspring parameters, and a 51% probability of not randomly updating a parameter in a group of offspring parameters.

[0113] In one example, the camera intrinsic parameters in the third group of child parameters {camera intrinsic parameter n3, camera extrinsic parameter w1 relative to the ground, person’s position on the ground z2, “person, lying flat”, “beach”} are randomly updated to obtain the first group of variation parameters {camera intrinsic parameter 1.1×n3, camera extrinsic parameter w1 relative to the ground, person’s position on the ground z2, “person, lying flat”, “beach”}.

[0114] In another example, the background description words in the fourth group of child parameters {camera intrinsic parameter n3, camera external parameter w2 relative to the ground, person's position on the ground z2, "person, walking", "grass"} are updated to obtain the second group of variation parameters {camera intrinsic parameter n3, camera external parameter w2 relative to the ground, person's position on the ground z2, "person, walking", "bedroom"}.

[0115] Based on the above embodiment, by introducing a preset disturbance probability, at least one parameter in each group of child parameters is randomly updated to generate corresponding mutation parameters. This mutation operation not only increases the diversity of parameter combinations, but also introduces new search directions for subsequent optimization processes. Whether it is fine-tuning of numerical type parameters or replacement of text type parameters, it enriches the parameter space to a certain extent and improves the flexibility and efficiency of parameter optimization.

[0116] refer to Figure 8As shown, it is a flow chart of an image generation method provided by another embodiment of the present application. Specifically, the image generation method may include the following process: determining the parameter space of the generator, the parameter space including generation parameters, such as rendering parameters and SD parameters. Among them, the SD parameters may include character description words and background description words obtained based on real images. Randomly generate multiple groups (recorded as n groups) of generation parameters to provide a basis for subsequent image generation. Randomly select a real image from a real image data set prepared in advance. Based on the SD image generation technology, the background is generated using the selected real image. According to the rendering parameters, the positions of the characters and the camera are arranged in the generated background. After the scene arrangement is completed, the rendering process is performed to generate a preliminary image containing the characters and the background. The preliminary generated characters are regenerated and optimized using the SD parameters to ensure their quality. The regenerated characters are synthesized with the previous background to form a complete image (such as the first image). The generated image and the real image in the data set are sent to the discriminator for training, and the discriminator will output a probability value representing the authenticity, indicating the similarity between the generated image and the real data set. After the model prediction, the images whose probability of generating the image exceeds the preset threshold k are screened out. If the proportion of images that meet the probability of exceeding the preset threshold k exceeds m, the number of these generation parameters is considered to meet the requirements. If the proportion of images that meet the probability of exceeding the preset threshold k does not exceed m, the generation parameters of images with a probability exceeding the preset threshold k are regenerated and optimized with the help of a genetic algorithm until enough generation parameters are obtained.

[0117] The image generation method provided in the embodiment of the present application determines the parameter space of the generator, including rendering parameters and character and background descriptors (SD parameters) extracted based on real images, randomly generates multiple sets of generation parameters, generates the background using SD image generation technology, and arranges the characters and camera positions for rendering according to the rendering parameters. The character image is optimized using SD parameters, and a complete image is synthesized with the background. The generated image is sent to the discriminator for training together with the real image. By screening and optimizing the generation parameters, it can be ensured that enough images with high authenticity can be generated based on the generation parameters, where the authenticity is measured by the probability value output by the discriminator. In this way, a large amount of annotated human body data can be automatically generated, the authenticity and diversity of the generated images can be improved, and the distribution of the generated data and the real data set can be guaranteed to be consistent.

[0118] refer to Fig. 9 As shown, it is a flow chart of a method for determining the authenticity of a first image provided by an embodiment of the present application. The first image in the generated data set generated by the generator and the real image in the real image data set can be sent to the discriminator for training, and the discriminator will output the probability (for example, ranging from 0 to 1) that the image comes from the real image data set, and the probability can be used as the authenticity of the first image.

[0119] Fig.10 It is a structural diagram of an image generating device provided in one embodiment of the present application.

[0120] In some embodiments, the image generating device 90 may include a plurality of functional modules composed of computer program segments. The computer programs of the various program segments in the image generating device 90 may be stored in a memory of an electronic device and executed by at least one processor to perform (see Figure 2 Description) Image generation function.

[0121] In this embodiment, the image generating device 90 can be divided into a plurality of functional modules according to the functions performed. The functional modules may include: a construction module 901, a generation module 902, a determination module 903, and an update module 904. The module referred to in this application refers to a series of computer program segments that can be executed by at least one processor and can complete fixed functions, which are stored in a memory. In this embodiment, the functional implementation of each module in the image generating device 90 can refer to the above definition of the image generating method, and will not be repeated here.

[0122] The construction module 901 is used to construct an image generation parameter set based on a real image set, wherein the image generation parameter set includes multiple groups of generation parameters, and each group of generation parameters corresponds to a real image in the real image set.

[0123] The generating module 902 is configured to generate a corresponding first image based on each set of generating parameters.

[0124] The determination module 903 is used to determine the authenticity of the first image, and select the first image corresponding to the authenticity greater than a preset threshold as the second image.

[0125] The updating module 904 is configured to update the image generation parameter set based on a genetic algorithm to obtain an updated image generation parameter set if the number of the second images is less than a preset number.

[0126] The generating module 902 is further configured to generate a target image based on the updated image generating parameter set.

[0127] An embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions. The method implemented when the program instructions are executed can refer to the methods in the above-mentioned embodiments of the present application.

[0128] The computer-readable storage medium may be an internal memory of the electronic device described in the above embodiment, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device.

[0129] In some embodiments, the computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the electronic device, etc.

[0130] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0131] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0132] In the embodiments provided in the present application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0133] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0134] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. An image generation method, characterized in that: The method comprises: Constructing an image generation parameter set based on a real image set, wherein the image generation parameter set includes multiple groups of generation parameters, and each group of generation parameters corresponds to a real image in the real image set; generating a corresponding first image based on each set of generation parameters; Determine the authenticity of the first image, and select the first image corresponding to the authenticity greater than a preset threshold as the second image; If the number of the second images is less than a preset number, updating the image generation parameter set based on a genetic algorithm to obtain an updated image generation parameter set; A target image is generated based on the updated set of image generation parameters.

2. The image generation method according to claim 1, characterized in that: Each set of generation parameters includes character generation parameters and background generation parameters. The image generation parameter set constructed based on the real image set includes: Randomly selecting a real image from the real image set, obtaining character description words in the selected real image through the multimodal large model, and determining corresponding character generation parameters according to the character description words; Background description words in the real image are obtained through the multimodal large model, and corresponding background generation parameters are determined according to the background description words.

3. The image generation method according to claim 2, characterized in that: The character generation parameters include rendering parameters and optimization parameters, wherein the rendering parameters are used to render and obtain a rendered image of the character; the optimization parameters include the character description words, and the optimization parameters are used to optimize the rendered image to obtain a character image.

4. The image generation method according to claim 1, characterized in that: Each set of generation parameters includes character generation parameters and background generation parameters; The generating a plurality of first images based on the plurality of groups of generation parameters comprises: Generate a character image based on the character generation parameters and the corresponding real image; Generate a background image based on the background generation parameters and the corresponding real image; The character image and the background image are fused to obtain the first image.

5. The image generation method according to claim 1, characterized in that: Determining the authenticity of the first image includes: The authenticity of the first image is determined by a pre-trained discriminant model, where the authenticity is used to indicate the probability that the first image is a real image.

6. The image generation method according to claim 5, characterized in that: The method further comprises: Obtaining a fused image based on the first image and the corresponding real image; According to the real image and the fused image, supervised training is performed on a preset neural network model to obtain the discriminant model.

7. The image generation method according to claim 6, characterized in that: The obtaining of a fused image based on the first image and the corresponding real image comprises: The real image is subjected to character cutout according to a preset preprocessing probability, and the character obtained by the cutout is fused into the background image corresponding to the first image to obtain the fused image.

8. The image generation method according to claim 1, characterized in that: If the number of the second images is less than a preset number, updating the image generation parameter set based on a genetic algorithm to obtain an updated image generation parameter set includes: Selecting multiple groups of generation parameters corresponding to the second image from the multiple groups of generation parameters, using the selected multiple groups of generation parameters as multiple groups of parent parameters, and determining the weight of a corresponding group of parent parameters according to the authenticity of each second image; Selecting a parent parameter from the plurality of groups of parent parameters according to the weight, and updating the selected parent parameter based on the genetic algorithm; The updated image generation parameter set is constructed according to the updated parent parameters.

9. The image generation method according to claim 8, characterized in that: The step of selecting a parent parameter from the plurality of groups of parent parameters according to the weight, and updating the selected parent parameter based on the genetic algorithm comprises: Randomly selecting two groups of parent generation parameters from the multiple groups of parent generation parameters based on the weights, cross-replacing the parameters in the two selected groups of parent generation parameters according to a preset crossover probability, to obtain two groups of child generation parameters, and using each group of child generation parameters as a group of updated parent generation parameters; The parameters in each group of offspring parameters are randomly updated according to a preset disturbance probability to obtain mutation parameters corresponding to each group of offspring parameters, and each group of mutation parameters is used as a group of updated parent parameters.

10. The image generation method according to claim 8, characterized in that: The method further comprises: If the number of updated parent parameters is less than the preset number, continue to select parent parameters from the multiple groups of parent parameters according to the weights, and update the continuously selected parent parameters until the number of updated parent parameters obtained is greater than or equal to the preset number.

11. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the image generation method according to any one of claims 1 to 10.

Images