Image generation method, system, apparatus, and storage medium

Abstract

The application discloses an image generation method, which comprises the following steps: providing an image feature option on a front-end page, so that a user can select an image feature prompt word; generating and displaying a plurality of reference images according to the image feature prompt word; obtaining a specified image based on a selection operation of the user from the plurality of reference images; segmenting the specified image according to a plurality of detail masks; and performing detail adjustment on the specified image according to a detail feature prompt word selected by the user and a corresponding detail mask, so as to obtain a final image. The application also discloses an image generation system, an electronic device and a computer readable storage medium. Therefore, the operation process of the user for generating an image can be simplified, the operation threshold is reduced, a plurality of outputs are provided for one request, the use efficiency is improved, the user is allowed to perform detail adjustment on the generated image, the other areas of the image are protected by the detail mask and the blooming situation is avoided.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image generation method, system, electronic device, and computer-readable storage medium. Background Technology

[0002] With the popularization and development of computer technology, users are using social media platforms more and more frequently. Nowadays, users want to showcase their uniqueness on social media platforms, and a personalized profile picture is an important part of this uniqueness. Users can describe the characteristics of their desired profile picture, and then image generation technology can automatically generate a corresponding profile picture. However, existing image generation technologies generally suffer from drawbacks such as high operational barriers, low efficiency, and uncontrollable generation. Summary of the Invention

[0003] The main objective of this application is to provide an image generation method, system, electronic device, and computer-readable storage medium, which aims to solve at least one of the aforementioned technical problems.

[0004] To achieve the above objectives, embodiments of this application provide an image generation method, the method comprising:

[0005] Provide image feature options on the front-end page so that users can select image feature prompts;

[0006] Multiple benchmark images are generated and displayed based on the image feature prompts;

[0007] The specified image is obtained based on the user's selection operation from the plurality of reference images;

[0008] Multiple detail masks are obtained based on the specified image segmentation;

[0009] Based on the user-selected detail feature hints and corresponding detail masks, the specified image is adjusted to obtain the final image.

[0010] Optionally, obtaining multiple detail masks based on the specified image segmentation includes:

[0011] The specified image is segmented at different locations using an image segmentation model, generating multiple detail masks corresponding to different locations.

[0012] Optionally, the image segmentation model includes at least one segmentation unit and a mask output unit, each segmentation unit including at least one feature extraction module and at least one attention module, the attention module being located after the feature extraction module; the step of segmenting different locations of the specified image using the image segmentation model to generate multiple detail masks corresponding to different locations includes:

[0013] In each segmentation unit, the feature extraction module extracts detailed features at a specified location from the specified image;

[0014] During the process of the feature extraction module extracting the detailed features of the location, the attention module increases the weight of the effective feature channel, which is the feature channel corresponding to the detailed features of the location.

[0015] The mask output unit generates a detail mask based on the detailed features of the location extracted by each segmentation unit, thereby obtaining multiple detail masks corresponding to different locations of the specified image.

[0016] Optionally, the training process of the image segmentation model includes:

[0017] Each image in the collected image dataset is annotated with details at different locations;

[0018] The image dataset and corresponding detail annotations are input into a pre-trained model for training. The pre-trained model learns to extract detail features from different locations in the image and generate corresponding detail masks, thereby obtaining the image segmentation model.

[0019] Optionally, generating and displaying multiple reference images based on the image feature prompts includes:

[0020] Receive the target prompt words selected by the user for each image feature;

[0021] Add the target prompt word to a preset prompt word template to generate complete prompt text;

[0022] Based on the complete prompt text, multiple corresponding baseline images are generated using a generative artificial intelligence model;

[0023] The multiple reference images are displayed for the user to select from.

[0024] Optionally, the step of adjusting the details of the specified image based on the user-selected detail feature prompts and corresponding detail masks includes:

[0025] Provide users with detailed feature options and obtain the detailed feature prompts selected by the user;

[0026] Obtain the target detail mask based on the detailed feature hints;

[0027] Based on the specified image, the detail feature cue words, and the target detail mask, a generative artificial intelligence model generates an image with adjusted details.

[0028] Optionally, generating the detail-adjusted image using a generative artificial intelligence model based on the specified image, the detail feature cue words, and the target detail mask includes:

[0029] Based on the target detail mask, the detail feature data at the corresponding position in the specified image is replaced with the detail feature data in the detail feature prompt, wherein the target detail mask is used to restrict the editable range in the specified image to the area corresponding to the position.

[0030] Furthermore, to achieve the above objectives, embodiments of this application also provide an image generation system, the system comprising:

[0031] The selection module is used to provide image feature options on the front-end page, allowing users to select image feature prompts;

[0032] The generation module is used to generate and display multiple reference images based on the image feature prompts;

[0033] The output module is used to obtain a specified image based on the user's selection operation from the plurality of reference images;

[0034] The segmentation module is used to segment the specified image to obtain multiple detail masks;

[0035] The adjustment module is used to perform detail adjustments on the specified image based on the detail feature prompts selected by the user and the corresponding detail mask, so as to obtain the final image.

[0036] To achieve the above objectives, this application also provides an electronic device, which includes: a memory, a processor, and an image generation program stored in the memory and executable on the processor. When the image generation program is executed by the processor, it implements the image generation method as described above.

[0037] To achieve the above objectives, embodiments of this application also provide a computer-readable storage medium storing an image generation program, which, when executed by a processor, implements the image generation method as described above.

[0038] The image generation method, system, electronic device, and computer-readable storage medium proposed in this application can provide image feature filtering options on the front-end page. Users only need to select some feature prompts to obtain the desired image, simplifying the image generation process and lowering the operational threshold. Furthermore, multiple outputs can be provided for a single request; that is, after selecting one prompt, users can obtain multiple benchmark images and then select the most satisfactory one, improving efficiency. Additionally, users can perform detailed adjustments on the generated image, using detail masks to protect other areas of the image from editing, avoiding smudging and improving the image generation effect. Attached Figure Description

[0039] Figure 1 An application environment architecture diagram for implementing various embodiments of this application;

[0040] Figure 2 This is a flowchart of an image generation method proposed in the first embodiment of this application;

[0041] Figure 3 This is a schematic diagram of a front-end page in this application;

[0042] Figure 4 for Figure 2 A detailed flowchart of step S202;

[0043] Figure 5 for Figure 2 A detailed flowchart of step S206;

[0044] Figure 6 for Figure 2 A detailed flowchart of step S208;

[0045] Figure 7 This is a schematic diagram illustrating the detailed adjustment of a specified image on the front-end page;

[0046] Figure 8 This is an illustration of generating a new image based on detail feature cues, using the specified image and the target detail mask.

[0047] Figure 9 This is a schematic flowchart of another form of the image generation method proposed in the first embodiment of this application;

[0048] Figure 10 This is a schematic diagram of the hardware architecture of an electronic device according to the second embodiment of this application;

[0049] Figure 11 This is a schematic diagram of the modules of an image generation system proposed in the third embodiment of this application. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0051] It should be noted that the descriptions involving "first," "second," etc., in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. Furthermore, the technical solutions of the various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.

[0052] The following provides an explanation of the terms used in this application:

[0053] AIGC (Artificial Intelligence Generated Content) refers to technologies based on generative adversarial networks, large-scale pre-trained models, and other artificial intelligence methods. It learns from and recognizes existing data to generate relevant content with appropriate generalization capabilities. The core idea of ​​AIGC is to use artificial intelligence algorithms to generate content with a certain degree of creativity and quality. Through training models and learning from large amounts of data, AIGC can generate relevant content based on input conditions or guidance. For example, by inputting keywords, descriptions, or samples, AIGC can generate matching articles, images, audio, etc.

[0054] Stable Diffusion: A deep learning text-to-image generation model, a type of AIGC model, primarily used to draw imagined scenes based on text descriptions. It employs a more stable and controllable diffusion process, thus generating high-quality images. By inputting a text prompt, the model outputs an image that matches the prompt.

[0055] Prompt: A prompt word is the input to the AIGC model used to draw the image. Generally, prompts are divided into positive prompt words and negative prompt words.

[0056] Image-based image generation (APIG) is a method of image generation using an AIGC model. Unlike methods that only use text prompts, APIG uses both a base image and text prompts as input, and then performs secondary creation based on the original image.

[0057] Smudge effect: This refers to the phenomenon where one text prompt word interferes with other prompt words. For example, if you input "redhair" as the prompt word, the generated image will also show red eyes.

[0058] Mask: Represented as a black and white image matrix, its function is to protect the image areas that do not need to be edited without destroying the original image (the black areas are protected).

[0059] Image segmentation model: A pixel-level image classification model. It is trained on a U-NET pre-trained model with a labeled dataset. This image segmentation model is used to generate detail masks (local masks) for test images, such as eyes, mouth, and hair.

[0060] U-NET: A neural network model whose core idea is to introduce skip connections, which greatly improves the accuracy of image segmentation. The main structure of the U-NET network includes three parts: decoder, encoder, and bottleneck layer.

[0061] AIGC is a commonly used image generation technology. AIGC image generation boasts advantages such as automation and high efficiency, rich creativity and personalization, high quality and customizability, and low cost. Currently, AIGC image generation is primarily based on StableDiffusion. While StableDiffusion is the most popular image generation method, it suffers from drawbacks for ordinary users, including a high learning curve and uncontrollable generation.

[0062] The technical solutions of this application will be described in detail below with reference to various embodiments.

[0063] Please see Figure 1 , Figure 1 This is an application environment architecture diagram for implementing various embodiments of this application. This application can be applied to application environments including, but not limited to, client 2, server 4, and network 6.

[0064] The client 2 is used to display the current application interface to the user and receive user prompts and selections. The client 2 can be a terminal device such as a PC (Personal Computer), mobile phone, tablet computer, portable computer, or wearable device.

[0065] The server 4 provides data and technical support to the client 2, such as providing users with image feature prompts, generating multiple reference images based on the user-selected prompts, obtaining a specified image based on the user-selected reference images, generating a detail mask based on the specified image, and adjusting image details based on the detail feature prompts. The server 4 can be a rack server, blade server, tower server, or cabinet server, and can be a standalone server or a server cluster composed of multiple servers.

[0066] The network 6 can be an intranet, the Internet, Global System for Mobile Communication (GSM), Wideband Code Division Multiple Access (WCDMA), 4G network, 5G network, Bluetooth, Wi-Fi, or other wireless or wired networks. The server 4 and one or more clients 2 communicate via the network 6 to perform data transmission and interaction.

[0067] It is worth noting that the images described in the various embodiments of this application are particularly suitable for digital avatars on social media platforms.

[0068] Example 1

[0069] like Figure 2 The diagram shown is a flowchart of an image generation method proposed in the first embodiment of this application. It is understood that the flowchart in this method embodiment is not intended to limit the order of execution steps. As needed, some steps in the flowchart can be added to or removed. The method will now be described using the server as the execution entity.

[0070] The method includes the following steps:

[0071] The S200 provides image feature options on the front-end page, allowing users to select image feature prompts.

[0072] In this embodiment, a drop-down menu of various image features can be provided on the front-end page. Each image feature offers multiple preset prompts for selection, simplifying user operation. Users only need to select the corresponding prompt from the drop-down menu based on the desired image feature.

[0073] like Figure 3 The image shown is a schematic diagram of one type of front-end page. Figure 3In this application, the front-end page includes dropdown menus corresponding to multiple image features, from which users can select desired prompts. For example, the prompt for the image feature "hairstyle" can be "random," the prompt for the image feature "expression" can be "smile," and the prompt for the image feature "viewpoint" can be "half-body," etc.

[0074] S202, Generate and display multiple reference images based on the image feature prompts.

[0075] After the user selects the image feature prompt, multiple corresponding reference images can be generated based on the prompt, for example, four reference images can be generated. Figure 3 In the process, after selecting the target prompt word corresponding to each image feature through the drop-down box on the front-end page, you can click "Generate Now" to generate four reference images. Furthermore, clicking "Generate Now" again will cycle through the four new reference images.

[0076] For more details, please refer to the following: Figure 4 This is a detailed flowchart of step S202 described above. It is understood that this flowchart is not intended to limit the order of execution steps. Steps in this flowchart can be added to or removed as needed. In this embodiment, step S202 specifically includes:

[0077] S2020: Receive the target prompt words selected by the user for each image feature.

[0078] The system receives the prompt words corresponding to each image feature selected by the user from the drop-down box on the front-end page, and uses them as target prompt words.

[0079] S2022, Add the target prompt word to the preset prompt word template to generate complete prompt text.

[0080] Since the user has selected multiple target prompt words for various image features, they need to be integrated before being input into the model, and a complete prompt text is generated using the prompt word template.

[0081] S2024, Based on the complete prompt text, generate multiple corresponding reference images using the AIGC model.

[0082] The AIGC model can generate an image matching the input prompt text. In this embodiment, to improve efficiency, multiple outputs are provided for each complete prompt text request, that is, multiple corresponding reference images are output so that the user can choose the image they prefer.

[0083] The AIGC model can be the StableDiffusion model, or other models with text-to-image and image-to-image generation functions; no limitation is made here. This application uses the StableDiffusion model as an example for illustration.

[0084] S2026, Display the multiple reference images for the user to select.

[0085] Based on the multiple benchmark images output by the AIGC model, all of these benchmark images can be displayed to the user for selection. Figure 3 Below each of the reference images is a "Use Image" button. Users can select the corresponding reference image by clicking "Use Image".

[0086] Back Figure 2 S204, Based on the user's selection operation from the plurality of reference images, a specified image is obtained.

[0087] Based on the multiple reference images displayed, the user can select one that best suits their needs. A specified image is then obtained based on the user's selection. In this embodiment, the specified image can be used as the user's digital avatar on a social media platform.

[0088] S206, obtain multiple detail masks based on the specified image segmentation.

[0089] In this embodiment, if the user is still not satisfied after obtaining the specified image and has more detailed modification requirements, modifications can be made based on the specified image or the reference image using an image-to-image method, such as modifying facial expressions, hair color, and eye color. The following explanation will use modification based on the specified image as an example. To improve the accuracy of the modification and approximate the effect of face sculpting, before modification, an image segmentation model is first used to segment different locations of the specified image, generating multiple detail masks (local masks) corresponding to different locations.

[0090] The image segmentation model uses a UNet pre-trained network as its base network. UNet is a classic image segmentation network with advantages such as a simple network structure and excellent segmentation results, making it suitable for scenarios like portrait segmentation where simplicity and speed are critical. Building upon UNet, to improve pixel-level segmentation accuracy, a channel attention module is added between each convolutional network layer, enhancing segmentation performance without compromising model speed. This channel attention module, also known as SENet (Squeeze-and-Excitation Networks), automatically learns the importance of each feature channel to enhance useful channel features and suppress unimportant ones.

[0091] The image segmentation model's dataset collection primarily focuses on anime character portraits, employing two methods: direct generation using AIGC tools such as Stable Diffusion and Midjouney (an AI drawing tool), or collection using web crawlers. For anime character portrait datasets, the focus is typically on templates for hair, eyes, lips, background, and clothing. The training process for the image segmentation model includes: first, detailed annotation of each image in the collected dataset at different locations, for example, manually. Then, the image dataset and corresponding detailed annotations are input into a pre-trained model for training. This allows the pre-trained model to learn to extract detailed features from different locations within the image, generating corresponding detail masks, thus obtaining the image segmentation model to serve online needs.

[0092] Specifically, the image segmentation model includes at least one segmentation unit and a mask output unit. Each segmentation unit includes at least one feature extraction module and at least one attention module, with the attention module located after the feature extraction module. Further reading... Figure 5 This is a detailed flowchart of step S206 described above. It is understood that this flowchart is not intended to limit the order of execution steps. Steps in this flowchart can be added to or removed as needed. In this embodiment, step S206 specifically includes:

[0093] S2060, in each of the segmentation units, the feature extraction module extracts a detailed feature at a specified location from the specified image.

[0094] S2062, during the feature extraction module's extraction of detailed features at the location, the attention module increases the weight of the effective feature channels. The effective feature channels are the feature channels corresponding to the detailed features at the location.

[0095] S2064, the mask output unit generates a detail mask based on the detail features of the position extracted by each segmentation unit, thereby obtaining multiple detail masks corresponding to different positions of the specified image.

[0096] Back Figure 2 S208, based on the detailed feature prompts selected by the user and the corresponding detailed mask, perform detail adjustments on the specified image to obtain the final image.

[0097] When a user needs to further fine-tune the specified image, they only need to select the detail feature to be modified and the desired effect from the provided drop-down menu, that is, select the detail prompt (e.g., pupil color -> purple), to replace the corresponding detail feature (pupil color). The user does not need to change other existing prompts.

[0098] For more details, please refer to the following: Figure 6 This is a detailed flowchart of step S208 described above. It is understood that this flowchart is not intended to limit the order of execution steps. Steps in this flowchart can be added to or removed as needed. In this embodiment, step S208 specifically includes:

[0099] S2080 provides users with detailed feature options and obtains the detailed feature prompts selected by the user.

[0100] In this embodiment, the specified image can be fine-tuned via a front-end page. For example... Figure 7 The diagram illustrates a method for making detailed adjustments to a specified image on a front-end page. The front-end page provides users with detailed feature options, each with multiple suggested terms via a dropdown menu. Users simply need to select the corresponding suggested term from the dropdown menu based on the desired detailed feature.

[0101] It is worth noting that the detailed feature options can be the same as the image feature options, or a separate set of drop-down options can be set up.

[0102] S2082, Obtain the target detail mask based on the detailed feature prompt words.

[0103] When a specific detail feature of the specified image needs to be modified, the target detail mask corresponding to that detail feature is obtained. The target detail mask is represented as a black and white image matrix. This embodiment supports further adjustment of the selected area of ​​the target detail mask without altering other areas of the image. The function of the target detail mask is to protect the image areas that do not need editing (i.e., the black corresponding areas) without damaging the original image.

[0104] S2084, Based on the specified image, the detail feature cue words, and the target detail mask, generate an image with adjusted details using the AIGC model.

[0105] The target detail mask is used to restrict the editable area in the specified image to the region corresponding to the location to be modified. Since the target detail mask can protect areas in the image that do not need to be edited, based on the target detail mask, the detail feature data at the corresponding location in the specified image can be replaced using the detail feature data in the detail feature hints, without affecting other areas of the specified image and without causing smudging.

[0106] like Figure 8 The image shown is a schematic diagram illustrating the generation of a new image based on the specified image and the target detail mask, according to the detail feature cue words. Figure 8 Based on the prompts "Smile" or "Cry" for the detailed feature "emoji," new images with the corresponding emojis can be generated. These new images are the final generated images and can be used as the user's digital avatar on social media platforms.

[0107] In one alternative embodiment, in addition to providing users with dropdown options for the image features or the detail features on the front-end page, custom prompts input by the user can also be received. These custom prompts can be used to generate the baseline image or to generate the final image in combination with a corresponding detail mask. Furthermore, the custom prompts can be positive or negative. Additionally, the custom prompts can be part of the image feature prompts or the detail feature prompts, and can be used in conjunction with prompts selected by the user from the options provided by the platform, all of which are applied to the AIGC model.

[0108] like Figure 9 The diagram shown illustrates another form of the image generation method. Figure 9 In this process, after the user selects image feature hints, multiple reference images are generated using the StableDiffusion model to generate raw images for the user to choose from. Personalization is also provided. The specified image (reference image) is segmented using an image segmentation model to obtain multiple detail masks. Then, based on the detail hints selected by the user, a new image is generated using the StableDiffusion model to generate raw images. The final generated image, as the image that satisfies the user, can be saved to a database or used directly. The specific implementation process of each of the above steps can be found in the relevant descriptions of the first or second embodiment, and will not be repeated here.

[0109] The image generation method proposed in this embodiment provides image feature filtering options on the front-end page. Users only need to select some feature prompts to obtain the desired image, simplifying the image generation process and lowering the operational threshold. Furthermore, this embodiment can provide multiple outputs for a single request; that is, after selecting one prompt, the user can obtain multiple baseline images and then select the most satisfactory one, improving efficiency and enhancing the user experience. In addition, this embodiment allows users to make detailed adjustments to the image. Using an image segmentation model, a detail mask is generated based on the user-selected image. Further modifications to the selected area of ​​the mask are supported without altering other areas of the image, avoiding smudging and improving the image generation effect.

[0110] Example 2

[0111] like Figure 10 The diagram shown illustrates the hardware architecture of an electronic device 20 according to a second embodiment of this application. In this embodiment, the electronic device 20 may include, but is not limited to, a memory 21, a processor 22, and a network interface 23, which are interconnected via a system bus. It should be noted that... Figure 10 Only the electronic device 20 with components 21-23 is shown; however, it should be understood that implementation of all shown components is not required, and more or fewer components may be implemented alternatively. In this embodiment, the electronic device 20 may be the server.

[0112] The memory 21 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 21 may be an internal storage unit of the electronic device 20, such as the hard disk or memory of the electronic device 20. In other embodiments, the memory 21 may also be an external storage device of the electronic device 20, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the electronic device 20. Of course, the memory 21 may include both the internal storage unit and the external storage device of the electronic device 20. In this embodiment, the memory 21 is typically used to store the operating system and various application software installed on the electronic device 20, such as the program code of the image generation system 60. In addition, the memory 21 can also be used to temporarily store various types of data that have been output or will be output.

[0113] In some embodiments, the processor 22 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 22 is typically used to control the overall operation of the electronic device 20. In this embodiment, the processor 22 is used to run program code stored in the memory 21 or process data, for example, to run the image generation system 60.

[0114] The network interface 23 may include a wireless network interface or a wired network interface, which is typically used to establish a communication connection between the electronic device 20 and other electronic devices.

[0115] Example 3

[0116] like Figure 11 The diagram shown is a modular schematic of an image generation system 60 according to a third embodiment of this application. The image generation system 60 can be divided into one or more program modules. One or more program modules are stored in a storage medium and executed by one or more processors to complete the embodiments of this application. The program modules referred to in the embodiments of this application are a series of computer program instruction segments capable of performing specific functions. The following description will specifically introduce the functions of each program module in this embodiment.

[0117] In this embodiment, the image generation system 60 includes:

[0118] Module 600 is used to provide image feature options on the front-end page for users to select image feature prompts.

[0119] The generation module 602 is used to generate and display multiple reference images based on the image feature prompts.

[0120] Output module 604 is used to obtain a specified image based on the user's selection operation from the plurality of reference images.

[0121] The segmentation module 606 is used to segment the specified image to obtain multiple detail masks.

[0122] The adjustment module 608 is used to perform detail adjustments on the specified image based on the detail feature prompts selected by the user and the corresponding detail mask to obtain the final image.

[0123] The specific implementation process of each of the above modules can be found in the description in the first embodiment above, and will not be repeated here.

[0124] The image generation system proposed in this embodiment provides image feature filtering options on the front-end page. Users only need to select some feature prompts to obtain the desired image, simplifying the image generation process and lowering the operational threshold. Furthermore, this embodiment can provide multiple outputs for a single request; that is, after selecting one prompt, the user can obtain multiple baseline images and then select the most satisfactory one, improving efficiency. In addition, this embodiment allows users to make detailed adjustments to the image. Using an image segmentation model, a detail mask is generated based on the user-selected image. Further modifications to the selected area of ​​the mask are supported without altering other areas of the image, avoiding smudging and improving the image generation effect.

[0125] Example 4

[0126] This application also provides another embodiment, namely, a computer-readable storage medium storing an image generation program that can be executed by at least one processor to cause the at least one processor to perform the steps of the image generation method as described above.

[0127] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0128] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0129] Obviously, those skilled in the art should understand that the modules or steps of the embodiments of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those presented here, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of this application are not limited to any particular combination of hardware and software.

[0130] The above are merely preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structural or procedural transformations made using the description and drawings of the present application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of the present application.

Claims

1. An image generation method, characterized in that, The method includes: Provide image feature options on the front-end page so that users can select image feature prompts; Multiple benchmark images are generated and displayed based on the image feature prompts; The specified image is obtained based on the user's selection operation from the plurality of reference images; Multiple detail masks are obtained based on the specified image segmentation; Based on the detailed feature prompts selected by the user and the corresponding detailed mask, the specified image is adjusted to obtain the final image; The step of obtaining multiple detail masks based on the specified image segmentation includes: The specified image is segmented at different locations using an image segmentation model, generating multiple detail masks corresponding to different locations; The image segmentation model includes at least one segmentation unit and a mask output unit. Each segmentation unit includes at least one feature extraction module and at least one attention module, with the attention module located after the feature extraction module. The step of segmenting different locations of the specified image using the image segmentation model to generate multiple detail masks corresponding to different locations includes: In each segmentation unit, the feature extraction module extracts detailed features at a specified location from the specified image; During the process of the feature extraction module extracting the detailed features of the location, the attention module increases the weight of the effective feature channel, which is the feature channel corresponding to the detailed features of the location. The mask output unit generates a detail mask based on the detailed features of the location extracted by each segmentation unit, thereby obtaining multiple detail masks corresponding to different locations of the specified image.

2. The image generation method according to claim 1, characterized in that, The training process of the image segmentation model includes: Each image in the collected image dataset is annotated with details at different locations; The image dataset and corresponding detail annotations are input into a pre-trained model for training. The pre-trained model learns to extract detail features from different locations in the image and generate corresponding detail masks, thereby obtaining the image segmentation model.

3. The image generation method according to claim 1, characterized in that, The step of generating and displaying multiple reference images based on the image feature prompts includes: Receive the target prompt words selected by the user for each image feature; Add the target prompt word to a preset prompt word template to generate complete prompt text; Based on the complete prompt text, multiple corresponding baseline images are generated using a generative artificial intelligence model; The multiple reference images are displayed for the user to select from.

4. The image generation method according to claim 1, characterized in that, The step of adjusting the details of the specified image based on the user-selected detail feature hints and corresponding detail masks includes: Provide users with detailed feature options and obtain the detailed feature prompts selected by the user; Obtain the target detail mask based on the detailed feature hints; Based on the specified image, the detail feature cue words, and the target detail mask, a generative artificial intelligence model generates an image with adjusted details.

5. The image generation method according to claim 4, characterized in that, The step of generating a detail-adjusted image using a generative artificial intelligence model based on the specified image, the detail feature cue words, and the target detail mask includes: Based on the target detail mask, the detail feature data at the corresponding position in the specified image is replaced with the detail feature data in the detail feature prompt, wherein the target detail mask is used to restrict the editable range in the specified image to the area corresponding to the position.

6. An image generation system, characterized in that, The system includes: The selection module is used to provide image feature options on the front-end page, allowing users to select image feature prompts; The generation module is used to generate and display multiple reference images based on the image feature prompts; The output module is used to obtain a specified image based on the user's selection operation from the plurality of reference images; The segmentation module is used to segment the specified image to obtain multiple detail masks; The adjustment module is used to perform detail adjustments on the specified image based on the detail feature prompts selected by the user and the corresponding detail mask to obtain the final image; The step of obtaining multiple detail masks based on the specified image segmentation includes: The specified image is segmented at different locations using an image segmentation model, generating multiple detail masks corresponding to different locations; The image segmentation model includes at least one segmentation unit and a mask output unit. Each segmentation unit includes at least one feature extraction module and at least one attention module, with the attention module located after the feature extraction module. The step of segmenting different locations of the specified image using the image segmentation model to generate multiple detail masks corresponding to different locations includes: In each segmentation unit, the feature extraction module extracts detailed features at a specified location from the specified image; During the process of the feature extraction module extracting the detailed features of the location, the attention module increases the weight of the effective feature channel, which is the feature channel corresponding to the detailed features of the location. The mask output unit generates a detail mask based on the detailed features of the location extracted by each segmentation unit, thereby obtaining multiple detail masks corresponding to different locations of the specified image.

7. An electronic device, characterized in that, The electronic device includes: a memory, a processor, and an image generation program stored in the memory and executable on the processor, wherein the image generation program, when executed by the processor, implements the image generation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image generation program, which, when executed by a processor, implements the image generation method as described in any one of claims 1 to 5.

9. A computer program product comprising computer instructions, characterized in that, When executed by a processor, the computer instructions implement the image generation method according to any one of claims 1 to 5.

Images