Image generation method and system, electronic equipment and storage medium

Abstract

The invention discloses an image generation method and system, electronic equipment and a storage medium. The method comprises the following steps: acquiring an initial model image; performing human body posture recognition and mask segmentation processing on the initial model image to obtain a mask image; generating a clothing text cue word according to the basic quality constraint parameter, the semantic enhancement parameter and the attribute combination generation parameter; performing image generation processing on the mask image through a generation model according to the clothing text cue word to obtain a target generated image; and carrying out anti-detection edge optimization processing on the target generation image to obtain a target model image. The image generation efficiency can be improved, and the method can be widely applied to the technical field of image processing.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image generation method, system, electronic device and storage medium. Background Art

[0002] With the rapid development of computer technology, image generation technology has been widely applied in various fields, particularly in e-commerce scenarios, where it can generate images of various products and models. Related technologies include face replacement and full-image generation, which replace the facial region of e-commerce model images or directly generate full-image images. However, in practical applications, these methods lack the ability to simulate local scene replacements such as clothing replacement and background modification. This makes it difficult to match the complex lighting and pose diversity requirements of e-commerce scenarios, affecting the efficiency of image generation.

[0003] In summary, the technical problems existing in the relevant technologies need to be improved. Summary of the Invention

[0004] The main purpose of the embodiments of the present application is to provide an image generation method, system, electronic device and storage medium, which can improve image generation efficiency.

[0005] To achieve the above objectives, an embodiment of the present application provides an image generation method, comprising:

[0006] Get the initial model image;

[0007] Performing human posture recognition and mask segmentation processing on the initial model image to obtain a mask image;

[0008] The clothing text prompt words are generated according to the basic quality constraint parameters, semantic enhancement parameters and attribute combination generation parameters;

[0009] Performing image generation processing on the mask image through a generative model according to the clothing text prompt word to obtain a target generated image;

[0010] Anti-detection edge optimization processing is performed on the target generated image to obtain a target model image.

[0011] In some embodiments, performing human posture recognition and mask segmentation processing on the initial model image to obtain a mask image includes the following steps:

[0012] Performing human posture recognition processing on the initial model image to obtain a human posture graph;

[0013] Performing mask segmentation processing on the initial model image to obtain a segmentation mask;

[0014] Mapping processing is performed on the segmentation mask between clothing parts and human body parts according to the human body posture graph to obtain the mask image.

[0015] In some embodiments, the step of generating clothing text prompt words according to the basic quality constraint parameters, the semantic enhancement parameters, and the attribute combination generation parameters comprises the following steps:

[0016] The basic quality constraint parameters are obtained according to the positive prompt engineering and negative prompt constraint construction;

[0017] Generating and processing clothing description sentences using a large language model according to a preset sentence template to obtain the semantic enhancement parameters;

[0018] Acquire a clothing data set, perform multi-dimensional description word combination processing on the clothing data set, and obtain the attribute combination generation parameter;

[0019] The basic quality constraint parameters, the semantic enhancement parameters and the attribute combination generation parameters are randomly sampled to obtain the clothing text prompt words.

[0020] In some embodiments, performing image generation processing on the mask image using a generative model according to the clothing text prompt word to obtain a target generated image includes the following steps:

[0021] Performing restoration processing on the initial model image according to the mask image to obtain a restored image;

[0022] Performing text encoding processing on the clothing text prompt words to obtain text embedding;

[0023] Performing feature extraction processing on the initial model image to obtain human body posture features;

[0024] Inputting the repaired image, the text embedding, the mask image and the human posture feature into the generative model for iterative denoising, and outputting an initial generated image;

[0025] The initial model image is fused with the initial generated image to obtain the target generated image.

[0026] In some embodiments, performing anti-detection edge optimization processing on the target generated image to obtain a target model image includes the following steps:

[0027] Performing labeling processing on the initial model image to obtain a three-color labeling image;

[0028] Performing edge optimization processing on the initial model image and the three-color marker image to obtain an updated mask;

[0029] Performing image synthesis processing on the initial model image and the target generated image according to the update mask to obtain the target model image.

[0030] In some embodiments, fusing the initial model image with the initial generated image to obtain the target generated image comprises the following steps:

[0031] performing region positioning processing on the initial model image according to the mask image to obtain background region pixels;

[0032] Performing region positioning processing on the initially generated image according to the mask image to obtain target region pixels;

[0033] The background area pixels and the target area pixels are fused to obtain the target generated image.

[0034] In some embodiments, after obtaining the target model image, the method further includes the following steps:

[0035] Performing detection processing on the target model image according to the authenticity detection model and the positioning model to obtain a detection image;

[0036] performing labeling processing on the detection image according to the mask image and the clothing text prompt word to obtain a labeled image;

[0037] Modular storage processing is performed on the initial model image and the detection image to obtain a generated image data set.

[0038] To achieve the above objectives, another aspect of the present application provides an image generation system, comprising:

[0039] The first module is used to obtain an initial model image;

[0040] The second module is used to perform human posture recognition and mask segmentation processing on the initial model image to obtain a mask image;

[0041] The third module is used to generate clothing text prompt words according to basic quality constraint parameters, semantic enhancement parameters and attribute combination generation parameters;

[0042] The fourth module is configured to perform image generation processing on the mask image through a generative model according to the clothing text prompt word to obtain a target generated image;

[0043] The fifth module is used to perform anti-detection edge optimization processing on the target generated image to obtain a target model image.

[0044] To achieve the above-mentioned purpose, another aspect of an embodiment of the present application provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor implements the above-mentioned method when executing the computer program.

[0045] To achieve the above objectives, another aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0046] The embodiments of the present application include at least the following beneficial effects: The present application provides an image generation method, system, electronic device and storage medium. The solution can generate a corresponding mask image based on posture guidance by performing human posture recognition and mask segmentation processing on the initial model image, can accurately control the generation area and optimize spatial alignment, and improve the generation quality and stability. In addition, the solution generates clothing text prompt words based on basic quality constraint parameters, semantic enhancement parameters and attribute combination generation parameters, and can fuse semantic enhancement and attribute combination sampling to generate corresponding prompt words, making the prompt words diversified and capable of generating images in multiple dimensions. In addition, the embodiments of the present application can controllably generate images with local scene replacement in multiple stages by integrating mask generation, prompt word generation and anti-detection optimization, thereby improving image generation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 This is a flowchart of an image generation method provided by an embodiment of the present application;

[0048] Figure 2 This is a flowchart of mask generation based on key point guidance provided by an embodiment of the present application;

[0049] Figure 3 This is a schematic diagram of a local scene replacement generation framework provided by an embodiment of the present application;

[0050] Figure 4 is a structural diagram of an image generation system provided in an embodiment of the present application;

[0051] Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present application. They are merely examples of systems and methods consistent with some aspects of the embodiments of the present application as detailed in the appended claims.

[0053] It will be understood that the terms "first", "second", etc. used in this application may be used herein to describe various concepts, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiments of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0054] The terms "at least one", "plurality", "each", "any", etc. used in this application include "at least one", "two" or more, "plurality" or "each", "any" or "any one", "each" or "any one" as used herein.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0056] Related technologies for artificial intelligence (AI)-generated image detection primarily rely on full-image generation datasets. These datasets primarily focus on face replacement or full-image generation scenarios, lacking targeted support for locally generated images. These methods utilize only a small number of generation methods to construct dataset samples, resulting in a single generation model. Furthermore, these generation methods lack simulation capabilities for local replacement scenarios, such as clothing replacement and background modification, making them difficult to adapt to the complex lighting and pose diversity requirements of e-commerce scenarios. This results in inadequate local generation and low scene realism.

[0057] In view of this, an embodiment of the present application provides an image generation method, system, electronic device and storage medium. By performing human posture recognition and mask segmentation processing on the initial model image, it is possible to generate a corresponding mask image based on posture guidance, accurately control the generation area and optimize spatial alignment, thereby improving the generation quality and stability. In addition, the scheme generates clothing text prompt words based on basic quality constraint parameters, semantic enhancement parameters and attribute combination generation parameters, and can fuse semantic enhancement and attribute combination sampling to generate corresponding prompt words, diversify the prompt words, and generate images in multiple dimensions. In addition, the embodiment of the present application integrates mask generation, prompt word generation and anti-detection optimization, and can controllably generate images with local scene replacement in multiple stages, thereby improving image generation efficiency.

[0058] The image generation method provided in the embodiment of the present application relates to the field of image processing technology. The image generation method provided in the embodiment of the present application can be applied to a terminal, can be applied to a server, or can be software running in a terminal or a server. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, and a car terminal, etc., but is not limited to this; the server side can be configured as an independent physical server, or can be configured as a server cluster or distributed system composed of multiple physical servers, and can also be configured as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application that implements the image generation method, etc., but is not limited to the above forms.

[0059] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0060] Figure 1This is an optional flowchart of the image generation method provided in the embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S105.

[0061] Step S101, obtaining an initial model image;

[0062] Step S102, performing human posture recognition and mask segmentation processing on the initial model image to obtain a mask image;

[0063] Step S103, generating clothing text prompt words according to the basic quality constraint parameters, the semantic enhancement parameters and the attribute combination generation parameters;

[0064] Step S104, performing image generation processing on the mask image through a generative model according to the clothing text prompt word to obtain a target generated image;

[0065] Step S105 , performing anti-detection edge optimization processing on the target generated image to obtain a target model image.

[0066] In steps S101 to S105 shown in the embodiment of the present application, an initial model image can be obtained by collecting and preprocessing the original data set, and the original data set can adopt an open source large-scale clothing data set. By performing human posture recognition on the initial model image, a mask can be segmented and generated based on posture guidance, thereby obtaining a mask image, which can be converted into a binary mask matrix. And according to the preset basic quality constraint parameters, semantic enhancement parameters and attribute combination generation parameters, clothing text prompt words can be generated, and the clothing text prompt words are used to guide and control the generation model to generate corresponding clothing images. In the embodiment of the present application, the image generation processing of local scene replacement is performed on the position corresponding to the mask image according to the clothing text prompt words guiding the generation model to obtain a target generated image. By performing anti-detection edge optimization on the target generated image, the edges of the generated image are optimized to obtain the final target model image.

[0067] One of the above technical solutions has the following advantages or beneficial effects: the embodiment of the present application can controllably generate images of local scene replacement in multiple stages by integrating mask generation, prompt word generation and anti-detection optimization, thereby improving image generation efficiency.

[0068] In step S101 of some embodiments, a large-scale clothing dataset is selected as the source dataset to ensure data diversity and quality, and then the dataset is cleaned, including resolution filtering, interference element detection, human key point verification, etc., and finally a model image that meets the conditions is screened out as the initial model image.

[0069] In step S102 of some embodiments, performing human posture recognition and mask segmentation processing on the initial model image to obtain a mask image includes the following steps:

[0070] Performing human posture recognition processing on the initial model image to obtain a human posture graph;

[0071] Performing mask segmentation processing on the initial model image to obtain a segmentation mask;

[0072] Mapping processing is performed on the segmentation mask between clothing parts and human body parts according to the human body posture graph to obtain the mask image.

[0073] In the examples of this application, please refer to Figure 2 , the human pose recognition model is used to perform human pose recognition on the initial model image to obtain a human pose graph. The human pose recognition model can adopt the OpenPose model, but is not limited to this. Then, the initial model image is subjected to mask segmentation processing by the segmentation model to obtain masks of different human body parts or clothing positions. Among them, the segmentation model can adopt the SAM model, but is not limited to this. According to the human pose graph, a mapping relationship between clothing parts and human body parts can be constructed, thereby generating a mask image containing the clothing parts position prior. The mask image can be represented as a binary mask matrix M∈{0,1} H×W Among them, H represents the image height, W represents the image width, and the matrix element M ij Representing the generated state (i∈[1,H],j∈[1,W]) of the pixel position (i,j), the expression of the matrix elements is as follows:

[0074]

[0075] It can be imagined that the reverse mask M can be obtained by matrix inversion operation ′ =1-M. In addition, the embodiment of the present application can also perform element-by-element inversion on the binary clothing mask image to obtain an inverse mask image, which can be used to guide the subsequent local generation model to retain the specified clothing area, thereby generating a background area.

[0076] One of the above technical solutions has the following advantages or beneficial effects: the embodiment of the present application generates a human body part mask through a posture key point heat map, can accurately map the clothing position, can accurately control the image area that needs to be replaced, and improves the efficiency of image generation.

[0077] In step S103 of some embodiments, the step of generating clothing text prompt words according to the basic quality constraint parameters, the semantic enhancement parameters, and the attribute combination generation parameters includes the following steps:

[0078] The basic quality constraint parameters are obtained according to the positive prompt engineering and negative prompt constraint construction;

[0079] Generating and processing clothing description sentences using a large language model according to a preset sentence template to obtain the semantic enhancement parameters;

[0080] Acquire a clothing data set, perform multi-dimensional description word combination processing on the clothing data set, and obtain the attribute combination generation parameter;

[0081] The basic quality constraint parameters, the semantic enhancement parameters and the attribute combination generation parameters are randomly sampled to obtain the clothing text prompt words.

[0082] In an embodiment of the present application, basic quality constraint parameters can be obtained by constructing positive prompt engineering and negative prompt constraints. The positive prompt engineering integrates standardized quality descriptors, such as "best quality, HDR, UHD, 8K", etc. Negative prompt constraints set quality constraint items for the model, such as deformation, etc. Negative prompts are used to improve the quality of images generated by the model, which can avoid generating some image features that do not meet expectations, and determine the optimal combination weights of positive and negative prompts through orthogonal experiments to form a stable quality baseline. The embodiment of the present application generates and processes clothing description sentences through a large language model based on a preset sentence template. Specifically, 120 fashion description sentences can be generated by GPT-4, and a benchmark corpus is formed after manual verification. Each sentence follows the template:

[0083] "[Clothing component] is made of [material], designed with [pattern], presents [style] style, and is suitable for [season] scenes";

[0084] Each parameter is sampled and populated from the attribute dictionary of the clothing dataset to ensure semantic compliance. ChatGPT has been trained on a large amount of corpus and has strong object description generation capabilities. Specifically for the clothing localization task in this embodiment, ChatGPT can interact with it to randomly generate diverse descriptions that conform to fashion semantics, thereby obtaining semantically enhanced parameters.

[0085] The embodiment of the present application can also be used with a pre-trained language model, which can be a transformer model, an LSTM model, a GPT-2 model, etc. Taking the transformer model as an example, the transformer model includes an encoder and a decoder. The training process of the language model includes: obtaining a sample image, and inputting the sample image into the transformer model. The sample image is encoded by the encoder of the transformer model to obtain a hidden feature vector of the sample image, and the hidden feature vector of the sample image is decoded by the decoder of the transformer model so that the hidden feature vector of the sample image is matrix-multiplied with the reference word vector to obtain an image description segment, and a sample image description segment is obtained. The similarity between the sample image description segment and the reference image description segment is calculated by the loss function of the transformer model, the loss function of the transformer model is optimized according to the similarity, the model loss of the loss function is back-propagated, and the model parameters are continuously adjusted until the similarity is greater than or equal to the similarity threshold, and the optimization of the transformer model is stopped to obtain a language model that meets the requirements.

[0086] In addition, the embodiment of the present application performs multi-dimensional descriptive word combination processing on the clothing data set, by defining an 8-dimensional attribute space, such as A = {a1, ..., a8}, covering core dimensions such as top type, bottom style, texture pattern, etc., each dimension contains 30 fine-grained descriptive words (such as a1 = {T-shirt, shirt, sweater, ...}). By randomly selecting 1 descriptive word for each attribute to combine, a rich and varied clothing text description is generated. When text prompts are needed in the local generation process, the embodiment of the present application randomly selects one of the above three methods to generate text prompt words about clothing, and the three methods can also be used to generate clothing text prompt words, which can generate a rich and varied clothing text description. It is conceivable that in order to further optimize the generation effect, the embodiment of the present application adjusts the step size and noise level in the diffusion process according to the complexity of the generation task, thereby improving the generation efficiency while ensuring the generation quality.

[0087] One of the above technical solutions has the following advantages or beneficial effects: the embodiment of the present application can not only improve the diversity of generated images, but also make the generated images meet the diverse needs in actual e-commerce scenarios.

[0088] In some embodiments, performing image generation processing on the mask image using a generative model according to the clothing text prompt word to obtain a target generated image includes the following steps:

[0089] Performing restoration processing on the initial model image according to the mask image to obtain a restored image;

[0090] Performing text encoding processing on the clothing text prompt words to obtain text embedding;

[0091] Performing feature extraction processing on the initial model image to obtain human body posture features;

[0092] Inputting the repaired image, the text embedding, the mask image and the human posture feature into the generative model for iterative denoising, and outputting an initial generated image;

[0093] The initial model image is fused with the initial generated image to obtain the target generated image.

[0094] In an embodiment of the present application, a target generated image can be obtained by performing image generation processing on a mask image through a generative model according to clothing text prompt words. Semantic conditions can be injected through text prompts, and fine-grained semantic control can be achieved through text-image joint encoding to generate content that conforms to specific semantics. The selected generative models can include 6 models, namely SD v1.5, SD v2, SDXL, Kandinsky, LoRA, and FashionDiffusion. Specifically, the initial model image is repaired according to the mask image to obtain a repaired image, wherein the mask image marks the area to be generated and the area to be retained, and the pixels in the non-masked area of ​​the initial model image are retained by element-by-element multiplication, and the pixels in the masked area are set to zero, thereby obtaining a repaired image. The calculation formula of the repaired image is as follows:

[0095] I m =I⊙(1-M);

[0096] Where, I m Represents the restored image, I represents the initial model image, ⊙ represents element-wise multiplication, and M represents the mask image.

[0097] Then, the clothing text prompt words are encoded to obtain text embedding, and the initial model image is subjected to feature extraction to obtain human posture features. The expressions for text encoding and feature extraction are as follows:

[0098] T=E(P), K=O(I);

[0099] Where T represents text embedding, E represents text encoder, P represents clothing text prompt words, O represents OpenPose model, K represents human posture features, and I represents the initial model image.

[0100] The repaired image, text embedding, mask image and human posture features are input into the generative model for iterative denoising. The expression of iterative denoising is as follows:

[0101]

[0102] Where x t represents the current noise level, t represents the number of iterations, α t ,β t Represents the noise scheduling parameter, and z is a variable that follows a standard normal distribution. The generative model can output the initial generated image G after iteration. The expression of the initial generated image is as follows:

[0103] G=f(x t ∣I m ,T,M,K);

[0104] Among them, the initial generated image is the restored image area, and it needs to be fused with the initial model image to obtain the final target generated image.

[0105] It is also conceivable that the embodiments of the present application can also generate target images through a context-driven repair method without text guidance, specifically through self-supervised learning to achieve the mining of internal semantics of the image and autonomous filling of missing areas. The selected generation models can include MAT, LaMa, AOT-GAN, DeepFill v1, DeepFill v2, Global&Local, and LDM, a total of 7 models.

[0106] One of the above technical solutions has the following advantages or beneficial effects: the embodiment of the present application guides the generation model to generate images at the position corresponding to the mask image through clothing text prompt words, can accurately locate the generation area and generate images in multiple dimensions, thereby improving the accuracy and diversity of generation.

[0107] In step S104 of some embodiments, performing anti-detection edge optimization processing on the target generated image to obtain a target model image includes the following steps:

[0108] Performing labeling processing on the initial model image to obtain a three-color labeling image;

[0109] Performing edge optimization processing on the initial model image and the three-color marker image to obtain an updated mask;

[0110] Performing image synthesis processing on the initial model image and the target generated image according to the update mask to obtain the target model image.

[0111] In the embodiment of the present application, the edge area of ​​the generated image often has unnatural transitions, such as illumination jumps and texture discontinuities, which not only affects the visual effect of the generated image, but may also become a clue for tampering detection. In order to optimize the edge of the generated image, the embodiment of the present application uses four technologies, PyMatting, VitMatte, MGMatting and Gaussian blur, to optimize the edge of the target generated image. The initial model image is annotated to obtain a three-color marking map, and the mask is updated in combination with the edge optimization technology to generate an updated mask. The initial model image and the target generated image are synthesized by updating the position corresponding to the mask to obtain the target model image after edge optimization. The expression of the update mask is as follows:

[0112] M σ =f(I,T);

[0113] Among them, M σ Represents the updated mask. I represents the initial model image, T represents the three-color labeling map, and f represents the specific edge processing method.

[0114] One of the above technical solutions has the following advantages or beneficial effects: the embodiment of the present application can improve the visual effect of the generated image by optimizing the edges of the generated image.

[0115] In some embodiments, fusing the initial model image with the initial generated image to obtain the target generated image comprises the following steps:

[0116] performing region positioning processing on the initial model image according to the mask image to obtain background region pixels;

[0117] Performing region positioning processing on the initially generated image according to the mask image to obtain target region pixels;

[0118] The background area pixels and the target area pixels are fused to obtain the target generated image.

[0119] In the embodiment of the present application, by fusing the non-masked area pixels in the initial model image, i.e., the background area pixels, with the masked area pixels in the generated initial generated image, i.e., the target area pixels, the visual consistency and contextual coherence of the generated image can be ensured. The fusion formula is shown as follows:

[0120] I′=I⊙(1-M)+G⊙M;

[0121] Where I′ represents the target generated image.

[0122] One of the above technical solutions has the following advantages or beneficial effects: the embodiment of the present application obtains a target generated image by fusing the initial model image with the initial generated image, and can replace the clothing of the initial model image, thereby generating a partially replaced image, thereby improving the efficiency of diversified image generation.

[0123] In some embodiments, after obtaining the target model image, the method further includes the following steps:

[0124] Performing detection processing on the target model image according to the authenticity detection model and the positioning model to obtain a detection image;

[0125] performing labeling processing on the detection image according to the mask image and the clothing text prompt word to obtain a labeled image;

[0126] Modular storage processing is performed on the initial model image and the detection image to obtain a generated image data set.

[0127] In an embodiment of the present application, after generating the target model image, it is necessary to evaluate the quality of the generated image to ensure that the quality of the generated image meets the requirements. The target model image is detected and processed using an authenticity detection model and a positioning model. Representative detection methods are selected to construct an evaluation matrix. By unifying the training set and experimental conditions, the performance of these detection methods is comprehensively analyzed, including establishing key indicators such as detection accuracy and generalization, so that the target model image can be detected using the detection method. A fine-grained annotation system is then provided for the dataset. Each generated sample contains detailed label information, including a generated area mask, text prompts, generation parameters, and other prompt inputs. This label information provides rich data support for subsequent generated image detection and analysis. In addition, the embodiment of the present application performs modular storage processing on the initial model image and the detection image to obtain a generated image dataset. For each generated image and the original image, they are placed in a separate folder, including mask-related information and a metadata file (JSON format). The 150,000 images for each generation method are divided into 100 folders, each containing 1,500 images and a JSON file. The JSON file contains generation-related parameter information, and the image files are stored in lossless compressed PNG format.

[0128] The following is a detailed description of the embodiments of the present application with reference to specific application examples:

[0129] The embodiment of the present application can be applied to application scenarios such as image generation and image detection. The image generation method of the embodiment of the present application can generate a variety of generated images. Among them, taking the application scenario of e-commerce image generation as an example, please refer to Figure 3By selecting a large-scale clothing dataset as the basic data source, the data is filtered and clarified to collect raw data. These raw data are model images to be processed. By constructing a human posture heat map for the model image, and using the recognized human posture heat map as the key point to guide the mask generation of the model image, the corresponding mask image is obtained. The generation parameters are configured, including basic quality constraint parameters, semantic enhancement parameters, and attribute combination generation parameters. Based on the generation parameters, clothing text prompt words are constructed and input into a large language model for local image generation. That is, the corresponding area of ​​the mask image is replaced based on the prompt word to obtain the generated model image. The model image is also optimized for anti-detection edges to improve the authenticity of the generated image and increase the detection difficulty.

[0130] See also Figure 4 The present application also provides an image generation system that can implement the above-mentioned image generation method. The system includes:

[0131] The first module 401 is used to obtain an initial model image;

[0132] The second module 402 is configured to perform human posture recognition and mask segmentation processing on the initial model image to obtain a mask image;

[0133] The third module 403 is used to generate clothing text prompt words according to the basic quality constraint parameters, the semantic enhancement parameters and the attribute combination generation parameters;

[0134] The fourth module 404 is configured to perform image generation processing on the mask image using a generative model according to the clothing text prompt word to obtain a target generated image;

[0135] The fifth module 405 is configured to perform anti-detection edge optimization processing on the target generated image to obtain a target model image.

[0136] It can be understood that the contents of the above method embodiments are all applicable to the present system embodiments, the functions specifically implemented by the present system embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0137] The present application also provides an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-mentioned image generation method. The electronic device can be any smart terminal including a tablet computer, an in-vehicle computer, or the like.

[0138] It can be understood that the contents of the above method embodiments are applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0139] See also Figure 5 , Figure 5 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0140] The processor 501 may be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is configured to execute relevant programs to implement the technical solutions provided in the embodiments of the present application.

[0141] The memory 502 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 502 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 502 and is called by the processor 501 to execute the image generation method of the embodiments of this application.

[0142] Input / output interface 503, used to implement information input and output;

[0143] Communication interface 504, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0144] Bus 505 , which transmits information between various components of the device (e.g., processor 501 , memory 502 , input / output interface 503 , and communication interface 504 );

[0145] The processor 501 , the memory 502 , the input / output interface 503 and the communication interface 504 are connected to each other in communication within the device via a bus 505 .

[0146] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned image generation method is implemented.

[0147] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiment, the functions specifically implemented by the present storage medium embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0148] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0149] The embodiments of the present application provide an image generation method, system, electronic device, and storage medium. This solution, by performing human posture recognition and mask segmentation processing on the initial model image, can generate a corresponding mask image based on posture guidance, accurately control the generation area, and optimize spatial alignment, thereby improving the generation quality and stability. In addition, the solution generates clothing text prompt words based on basic quality constraint parameters, semantic enhancement parameters, and attribute combination generation parameters. It can fuse semantic enhancement and attribute combination sampling to generate corresponding prompt words, diversify the prompt words, and generate images in multiple dimensions. In addition, the embodiments of the present application, by integrating mask generation, prompt word generation, and anti-detection optimization, can controllably generate images with local scene replacement in multiple stages, thereby improving image generation efficiency.

[0150] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0151] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than shown in the figures, or a combination of certain steps, or different steps.

[0152] The system embodiment described above is merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0153] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0154] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0155] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0156] In the several embodiments provided in this application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the system embodiments described above are merely illustrative. For example, the division of the above units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0157] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0159] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0160] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. An image generation method, characterized in that: The method comprises the following steps: Get the initial model image; Performing human posture recognition and mask segmentation processing on the initial model image to obtain a mask image; The clothing text prompt words are generated according to the basic quality constraint parameters, semantic enhancement parameters and attribute combination generation parameters; Performing image generation processing on the mask image through a generative model according to the clothing text prompt word to obtain a target generated image; Anti-detection edge optimization processing is performed on the target generated image to obtain a target model image.

2. The method according to claim 1, characterized in that The step of performing human posture recognition and mask segmentation processing on the initial model image to obtain a mask image comprises the following steps: Performing human posture recognition processing on the initial model image to obtain a human posture graph; Performing mask segmentation processing on the initial model image to obtain a segmentation mask; Mapping processing is performed on the segmentation mask between clothing parts and human body parts according to the human body posture graph to obtain the mask image.

3. The method according to claim 1, characterized in that The method of generating clothing text prompt words according to the basic quality constraint parameters, the semantic enhancement parameters and the attribute combination generation parameters includes the following steps: The basic quality constraint parameters are obtained according to the positive prompt engineering and negative prompt constraint construction; Generating and processing clothing description sentences using a large language model according to a preset sentence template to obtain the semantic enhancement parameters; Acquire a clothing data set, perform multi-dimensional description word combination processing on the clothing data set, and obtain the attribute combination generation parameter; The basic quality constraint parameters, the semantic enhancement parameters and the attribute combination generation parameters are randomly sampled to obtain the clothing text prompt words.

4. The method according to claim 1, wherein The step of performing image generation processing on the mask image through a generative model according to the clothing text prompt word to obtain a target generated image comprises the following steps: Performing restoration processing on the initial model image according to the mask image to obtain a restored image; Performing text encoding processing on the clothing text prompt words to obtain text embedding; Performing feature extraction processing on the initial model image to obtain human body posture features; Inputting the repaired image, the text embedding, the mask image and the human posture feature into the generative model for iterative denoising, and outputting an initial generated image; The initial model image is fused with the initial generated image to obtain the target generated image.

5. The method according to claim 1, wherein The step of performing anti-detection edge optimization processing on the target generated image to obtain a target model image comprises the following steps: Performing labeling processing on the initial model image to obtain a three-color labeling image; Performing edge optimization processing on the initial model image and the three-color marker image to obtain an updated mask; Performing image synthesis processing on the initial model image and the target generated image according to the update mask to obtain the target model image.

6. The method according to claim 4, characterized in that The step of fusing the initial model image with the initial generated image to obtain the target generated image comprises the following steps: performing region positioning processing on the initial model image according to the mask image to obtain background region pixels; Performing region positioning processing on the initially generated image according to the mask image to obtain target region pixels; The background area pixels and the target area pixels are fused to obtain the target generated image.

7. The method according to any one of claims 1 to 6, characterized in that After obtaining the target model image, the method further includes the following steps: Performing detection processing on the target model image according to the authenticity detection model and the positioning model to obtain a detection image; performing labeling processing on the detection image according to the mask image and the clothing text prompt word to obtain a labeled image; Modular storage processing is performed on the initial model image and the detection image to obtain a generated image data set.

8. An image generation system, characterized in that: The system comprises: The first module is used to obtain an initial model image; The second module is used to perform human posture recognition and mask segmentation processing on the initial model image to obtain a mask image; The third module is used to generate clothing text prompt words according to basic quality constraint parameters, semantic enhancement parameters and attribute combination generation parameters; The fourth module is configured to perform image generation processing on the mask image through a generative model according to the clothing text prompt word to obtain a target generated image; The fifth module is used to perform anti-detection edge optimization processing on the target generated image to obtain a target model image.

9. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images