Image generation method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to an image generation method and device, equipment and a storage medium. The method comprises the steps of obtaining at least one personal image in response to a target interaction operation; the portrait image comprises one or more to-be-processed target objects; determining a reference group photo image, wherein the number of reference objects in the reference group photo image is the same as that of the target objects; based on the portrait image, fusing the target object to a reference object in the reference group photo image to generate an initial group photo image; based on a target diffusion model corresponding to the target object, adjusting at least one object feature of a fusion object corresponding to the target object in the initial group photo image, and generating a target group photo image; the target diffusion model is obtained by performing fine tuning training on the initial diffusion model based on an image sample of the target object. In this way, the flexibility and diversity of personalized image generation are improved, and the authenticity and image quality of the generated personalized image are improved.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and in particular to an image generation method, apparatus, device, and storage medium. Background Technology

[0002] With the development of image technology, personalized image generation solutions have emerged that generate images with certain styles and effects based on users' portrait images, in order to enhance the interest of images.

[0003] Currently, the aforementioned personalized image generation schemes are mainly model-based. However, this approach is primarily suitable for generating images of a single person. Extending it to generate images of multiple people can lead to confusion in the model's feature inference, resulting in low similarity between the generated image and the input image. This leads to poor realism and low image quality in the generated personalized images. Summary of the Invention

[0004] To address the aforementioned technical problems, this disclosure provides an image generation method, apparatus, device, and storage medium.

[0005] In a first aspect, embodiments of this disclosure provide an image generation method, the method comprising:

[0006] In response to a target interaction operation, at least one portrait image is acquired; wherein the portrait image contains one or more target objects to be processed;

[0007] A reference group photo image is determined; wherein the number of reference objects in the reference group photo image is the same as the number of target objects;

[0008] Based on the portrait image, the target object is fused into the reference object in the reference group photo image to generate an initial group photo image;

[0009] Based on the target diffusion model corresponding to the target object, at least one object feature of the fusion object corresponding to the target object in the initial group photo image is adjusted to generate the target group photo image; wherein, the target diffusion model is obtained by fine-tuning and training the initial diffusion model based on the image samples of the target object.

[0010] Secondly, embodiments of this disclosure also provide an image generation apparatus, the apparatus comprising:

[0011] A portrait image acquisition module is used to acquire at least one portrait image in response to a target interaction operation; wherein the portrait image contains one or more target objects to be processed;

[0012] A reference group photo image determination module is used to determine a reference group photo image; wherein the number of reference objects in the reference group photo image is the same as the number of target objects;

[0013] An initial group photo image generation module is used to generate an initial group photo image by fusing the target object into the reference object in the reference group photo image based on the portrait image;

[0014] The target group photo image generation module is used to adjust at least one object feature of the fusion object corresponding to the target object in the initial group photo image based on the target diffusion model corresponding to the target object, and generate the target group photo image; wherein, the target diffusion model is obtained by fine-tuning and training the initial diffusion model based on the image samples of the target object.

[0015] Thirdly, embodiments of this disclosure also provide an electronic device, the electronic device comprising:

[0016] processor;

[0017] Memory, used to store executable instructions;

[0018] The processor is configured to read executable instructions from memory and execute the executable instructions to implement the image generation method described in any embodiment of this disclosure.

[0019] Fourthly, embodiments of this disclosure also provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the image generation method described in any embodiment of this disclosure.

[0020] Fifthly, embodiments of this disclosure also provide a computer program product for executing the image generation method described in any embodiment of this disclosure.

[0021] The image generation method, apparatus, device, and storage medium of this disclosure can acquire portrait images containing multiple target objects to be processed and determine reference group photos containing the same number of reference objects. Then, each target object is initially fused into the reference group photo image to obtain an initial group photo image containing multiple fused objects. Next, a target diffusion model adapted to each target object and finely trained based on image samples of the target object is used to adjust at least one object feature of the fused object corresponding to the local target object in the initial group photo image. This significantly reduces the image problems of insufficient similarity and unnatural visual effects of the fused objects in the initial group photo image, resulting in the final target group photo image. On the one hand, it allows for the generation of personalized group photos by inputting portrait images with no limitation on the target objects, improving the flexibility and diversity of image generation. On the other hand, it increases the similarity between each object in the target group photo image and the corresponding target object, and enhances the naturalness of each object in the target group photo image, thereby improving the realism and image quality of personalized image generation.

[0022] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. Attached Figure Description

[0023] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0024] Figure 1 This is a schematic flowchart of an image generation method provided in an embodiment of the present disclosure;

[0025] Figure 2 for Figure 1 A schematic diagram of the refinement process in S130 of an image generation method is shown.

[0026] Figure 3 for Figure 1 A schematic diagram of the refinement process in S140 of an image generation method is shown.

[0027] Figure 4 A schematic diagram illustrating the processing steps of an image generation method using person image 1 and person image 2 as examples, provided in this embodiment of the disclosure;

[0028] Figure 5 This is a schematic diagram of the structure of an image generation apparatus provided in an embodiment of the present disclosure;

[0029] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0030] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0031] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0032] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0033] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0034] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0035] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0036] In related technologies, there are three main methods for generating group photos of multiple people. The first is a generative model-based image generation method, which uses a trained model to generate personalized portrait images. However, when extending this model to group photo generation, the model easily confuses the features of multiple people, resulting in low similarity between the people in the resulting group photo and the original input, and the generated people appear unnatural, leading to low image quality. The second method introduces a mask-based attention mechanism into the aforementioned model to mask information from other people during the group photo generation process. However, the group photos generated by this method still appear unnatural and have low quality; furthermore, it limits the content of the group photo, resulting in low flexibility and a lack of diversity. The third method fuses the features of the people in the input image into the group photo. However, this method requires multiple rounds of feature fusion, making the process complex, and the resulting group photo shows obvious fusion artifacts, resulting in poor naturalness of the people.

[0037] Based on the above, this disclosure provides an image generation scheme that, through a single processing step, initially fuses each target object in the input portrait image into a group photo image template (i.e., a reference group photo image) to obtain an initial group photo image. Then, using a target diffusion model adapted to each target object, at least one object feature in the corresponding local image of the initial group photo image is adjusted to obtain a more natural and clearer target group photo image. This approach not only avoids limiting the content of the group photo image, thus improving the flexibility and diversity of group photo image generation, but also enhances the naturalness and image quality of the group photo image.

[0038] The image generation method provided in this disclosure is applicable to scenarios where a group photo image is synthesized from an input image containing multiple objects. This method can be executed by an image generation device, which can be implemented in software and / or hardware and can be integrated into an electronic device with image processing capabilities. This electronic device may include, but is not limited to, smartphones, tablet PCs, personal digital assistants (PDAs), laptops, desktop computers, mobile workstations, or servers.

[0039] Figure 1 A schematic flowchart of an image generation method provided by an embodiment of this disclosure is shown. Figure 1 As shown, the image generation method may include the following steps:

[0040] S110. In response to a target interaction operation, acquire at least one portrait image; the portrait image contains one or more target objects to be processed.

[0041] The target interactive operation is the interactive operation that triggers the input of a portrait image, such as image upload, image selection, or image capture / shooting. The portrait image is the image of a person to be processed, containing one or more target objects. The target object is the object the user is interested in and wants to transfer to the group photo image; it can be a part of the person (such as facial features, limbs, etc.) or certain items worn by the person. The person / portrait here can be a real person or a computer-generated virtual person.

[0042] Specifically, the electronic device provides a group photo image generation function and an interactive interface for using this function. Users can perform targeted interactive operations based on the prompts in the interface. In response to the targeted interactive operation, the electronic device identifies the image obtained after the operation as the portrait image to be processed. This portrait image can be an image of a single person or an image containing multiple people. There can be one or multiple portrait images. For example, if a user wants to create a group photo of four people, they can input four single-person portrait images, two double-person portrait images, two single-person portrait images and one double-person portrait image, or one single-person portrait image and one triple-person portrait image, etc.

[0043] In some embodiments, S110 includes: in response to a second image selection operation, determining the selected image in the image library as a portrait image.

[0044] The selected image contains one or more target objects.

[0045] The second image selection operation is the interactive operation for selecting an image corresponding to the portrait image input function. The image library stores multiple images, which can be a photo album provided by the user or a candidate image library provided by the platform.

[0046] Specifically, the electronic device can identify the image selected by the user as a portrait image in response to a second image selection operation performed by the user. For example, if the user triggers a function button to select an image from the album and then performs the operation of selecting a photo from the album, the electronic device can identify each image selected by the user as the corresponding portrait image. As another example, to provide a group photo generation experience, the electronic device can provide several candidate portrait images for the user to choose from. In this case, the electronic device identifies the candidate portrait image selected by the user as a portrait image.

[0047] In other embodiments, S110 includes: in response to an image acquisition operation, determining the acquired image as a portrait image.

[0048] The acquired image contains one or more target objects.

[0049] Specifically, in addition to selecting existing images, users can also capture images in real time. For example, if a user wants to obtain a multi-person image with a certain artistic style, but the current shooting environment does not support that artistic style, they can capture images of multiple people in real time to obtain a single-person image or a multi-person image. In this case, the electronic device can identify the captured image as a portrait image.

[0050] S120. Determine a reference group photo image; the number of reference objects in the reference group photo image is the same as the number of target objects.

[0051] The reference group photo image is an image that provides a group photo image template function. The reference object is an object contained in the reference group photo image that is of the same type as the target object.

[0052] Specifically, the electronic device can determine a reference group photo image to provide other image features (such as overall composition, color scheme, lighting effects, etc.) before the target object when generating subsequent group photos. This reference group photo image can be an image uploaded / selected by the user, an image automatically generated by the electronic device according to the user's needs, or an image automatically selected by the electronic device from candidate group photo images provided in the background according to the user's needs, etc.

[0053] In some embodiments, S120 includes: in response to an image upload operation, determining the received group photo image as a reference group photo image.

[0054] Specifically, users can upload their desired group photo image. The electronic device then responds to the image upload operation by providing a reference group photo image based on the user's input.

[0055] In other embodiments, S120 includes: in response to the first image selection operation, determining the selected candidate group photo image as the reference group photo image.

[0056] The first image selection operation is an interactive operation that determines the image selection corresponding to the group photo template function. The candidate group photo images are pre-set group photo images.

[0057] Specifically, the electronic device can provide multiple candidate group photo images. When the user performs the first image selection operation, the electronic device can determine the candidate group photo image selected by the user as the reference group photo image.

[0058] It should be noted that whether a user uploads or selects a reference group photo, the system must verify that the number of reference objects matches the number of target objects. If they match, the process continues; if they do not match, the user is prompted that the reference group photo is incorrect, and / or the electronic device automatically generates a suitable reference group photo.

[0059] In some other embodiments, S120 includes: determining target text containing a description of the group photo; constructing an image generation prompt based on the target text and a second preset prompt; and calling an image generation model based on the image generation prompt to generate a reference group photo image.

[0060] The target text is the text used to describe the desired group photo image. The second preset prompt is a pre-built model prompt containing pre-designed text content, such as the process of generating the group photo image, the output format, etc. The image generation prompt is the final prompt used to trigger the model to generate the reference group photo image. The image generation model is a generative model with the ability to generate images from text, which can be pre-trained according to business needs.

[0061] Specifically, if the user has not specified a reference group photo image, the electronic device can generate one automatically. In this process, the electronic device first obtains the target text. Then, it combines this target text with a second preset prompt, such as filling the text content of the target text into the text placeholders in the second preset prompt, to generate an image generation prompt. Afterward, the electronic device inputs the image generation prompt into an image generation model, and through the model's processing, generates the reference group photo image.

[0062] In one example, "determining the target text containing the group photo description" in the above embodiment includes: in response to a text input operation, determining the received text as the target text.

[0063] Specifically, users can input target text according to their needs. In this case, the electronic device responds to the text input operation and identifies the received text as the target text.

[0064] In another example, "determining the target text containing the group photo description" in the above embodiment includes: generating the target text based on the object information of the target object.

[0065] Specifically, if the text entered by the user is inappropriate, or if the user does not wish to enter text or an image, the electronic device can automatically generate suitable target text based on the object information (such as the number, gender, style, etc. of the target object) in the user-input portrait image. For example, the electronic device can use the object information as input data, call a text generation model, and output the target text.

[0066] In yet another example, "determining the target text containing the group photo description" in the above embodiment includes: generating the target text based on the object information of the target object and the style of the target group photo.

[0067] The target group photo style refers to the style of the group photo image. This style can be determined based on user behavior data during the group photo generation process over a historical period. For example, it can be determined by analyzing previously uploaded reference group photos and positive or negative feedback from users regarding the final composite group photo (the target image). Alternatively, the target group photo style can be a user-defined preferred style. Here, style can refer to the overall composition of the image (spatial sense and hierarchy of elements), color, tone, lighting, and element type (such as architecture, natural scenery, or home furnishings).

[0068] Specifically, when automatically generating target text, electronic devices can consider not only the object information of the target object but also the style of the target group photo to further improve the relevance of the target text to user needs. For example, an electronic device can use object information and the style of the target group photo as input data, call a text generation model, and output the target text.

[0069] In some other embodiments, S120 includes: selecting a reference group photo image from a plurality of candidate group photo images based on object information of the target object.

[0070] Specifically, the electronic device can provide multiple candidate group photo images. If it is determined that the user has not manually determined the target text or reference group photo image, the electronic device can automatically select the candidate group photo image with the highest matching degree from all candidate group photo images as the reference group photo image, based on the degree of matching between the object information of the target object in the input image and the object information of the reference object in each candidate group photo image.

[0071] In some other embodiments, S120 includes: selecting a reference group photo image from multiple candidate group photo images based on object information of the target object and the target group photo style.

[0072] Specifically, during the process of electronic devices automatically selecting candidate group photos, in addition to the object information of the target object, the target group photo style that suits the user can also be considered to further improve the fit between the selected reference group photos and the user's needs.

[0073] S130. Based on the portrait image, the target object is fused into the reference object in the reference group photo image to generate an initial group photo image.

[0074] Specifically, the electronic device can extract the object features of the target object from the portrait image. Then, it performs image fusion processing on each object feature and the corresponding reference object in the reference group photo image to obtain a preliminary group photo image with the target objects fused (i.e., the initial group photo image). This image fusion process can complete the fusion processing of multiple target objects at once, reducing the number of fusion steps in the generation process of group photos with multiple people.

[0075] In some embodiments, S130 includes: generating a first object feature for each reference object based on a reference group photo image; extracting a second object feature for each target object from a portrait image; and fusing the target objects into the reference objects in the reference group photo image based on the first object feature and the second object feature to generate an initial group photo image.

[0076] The first object feature is the object feature of the reference object. This could be an object mask image, or a combination of the object mask image and the object category identifier of the reference object (i.e., the first object category identifier). The object mask image is a binary image where the image region containing the reference object has a value of 1, and other image regions have values ​​of 0. The second object feature is the object feature of the target object. This could be object description features describing the target object's structural key points, texture, edges, details, etc., or a combination of object description features and the target object's object category identifier (i.e., the second object category identifier). The object category identifier represents the object category to which an object belongs. The object category is a predefined category that distinguishes different objects and can be determined by analyzing multiple dimensions of the object's information. Taking people as an example, the object category can be a makeup style category, such as nude makeup, heavy makeup, workplace makeup, everyday makeup, cute makeup, mature makeup, etc.; the object category can also be a clothing category, such as a clothing category determined by skirts, high heels, etc., and another clothing category determined by suits, ties, etc.; it can also be a formal wear category, casual wear category, urban category, classic / ethnic / country category, etc.; the object category can also be different categories divided according to dimensions such as whether the facial structure is three-dimensional, whether there are wrinkles on the face, hair length, hair color, etc.

[0077] Specifically, the initial fusion process of the electronic device for target objects is as follows: First object features of each reference object are extracted from the reference group photo image, and second object features of each target object are extracted from the portrait image. Then, based on each first and second object feature, a correspondence between the target object and the reference object is established. Afterward, according to this correspondence, each target object is fused to the corresponding image region of the reference reference object in the reference group photo image, generating an initial group photo image. This allows for targeted object fusion, reducing the probability of object feature confusion to some extent, improving the accuracy of object fusion, and thus further enhancing the portrait similarity of the group photo image.

[0078] S140. Based on the target diffusion model corresponding to the target object, adjust at least one object feature of the fusion object corresponding to the target object in the initial group photo image to generate the target group photo image.

[0079] The target diffusion model is obtained by fine-tuning the initial diffusion model based on image samples of the target object. The initial diffusion model is a diffusion model obtained by pre-training using image samples of the target object. Object features refer to the identifiable attributes or characteristics of the fused object in the image. For example, object features can be the structural features, shape features or edge features, pose features, texture features, depth information features, lighting features or shadow features of the fused object, etc. Taking the target object as a person's head as an example, object features can be the distribution relationship features between the facial features and hair, skin texture features, head pose features, edge features between the face and hair, depth difference features of different parts of the head region, lighting and shadow relationship features of different parts of the head region, etc.

[0080] Specifically, due to limitations in image fusion technology, the reference object (i.e., the fused object) in the initial group photo image obtained above, which incorporates the object features of the target object, may exhibit unnatural mask-like appearance and insufficient image quality. Therefore, this embodiment of the present disclosure includes an image adjustment process to significantly reduce the aforementioned image problems.

[0081] Because the initial group photo image contains multiple fusion objects, and each fusion object corresponds to a different target object and has different object features, in order to improve the image restoration effect, the initial diffusion model can be fine-tuned for each target object to obtain a candidate diffusion model that is more suitable for the target object. Then, for each target object, the electronic device calls the candidate diffusion model (i.e., the target diffusion model) corresponding to the target object to adjust one or more object features in the image region where the fusion object is located in the initial fusion image, so that the fusion object has a high similarity to the target object in one or more object features, thereby improving the similarity between the adjusted fusion object and its corresponding target object, and making the fusion object (such as a person's face) maintain reasonable coherence and better visual naturalness with other objects in its local image (such as a person's hair, neck, or shoulder objects) in one or more object features, thereby improving the naturalness of the adjusted fusion object. After this processing, the electronic device can perform targeted image adjustments on each fusion object in the initial fusion image to obtain the target group photo image.

[0082] The image generation method provided in the above embodiments of this disclosure can acquire portrait images containing multiple target objects to be processed and determine reference group photos containing the same number of reference objects. Then, each target object is initially fused into the reference group photo image to obtain an initial group photo image containing multiple fused objects. Then, using a target diffusion model adapted to each target object and finely trained based on image samples of the target object, at least one object feature of the local fused object corresponding to the corresponding target object in the initial group photo image is adjusted to greatly reduce the image problems of insufficient similarity and unnaturalness of the fused objects in the initial group photo image, thereby obtaining the final target group photo image. On the one hand, it can generate personalized group photo images by inputting portrait images with no limitation on target objects, which improves the flexibility and diversity of image generation. On the other hand, it improves the similarity between each object contained in the target group photo image and the corresponding target object, and improves the naturalness of each object in the target group photo image, thereby improving the realism and image quality of personalized image generation.

[0083] Figure 2 This is provided by the embodiments of this disclosure. Figure 1 A detailed flowchart of step S130 in an image generation method is shown. (See attached flowchart.) Figure 2 As shown, when the first object feature includes an object mask image and a first object category identifier, and the second object feature includes an object description feature and a second object category identifier, S130 "based on the portrait image, fusing the target object into the reference object in the reference group photo image to generate an initial group photo image" includes the following steps:

[0084] S210. Perform object category recognition and mask segmentation on the reference objects in the reference group photo image respectively, and generate an object mask image and a first object category identifier for each reference object.

[0085] Specifically, the electronic device can perform mask segmentation processing on the reference group photo image to obtain an object mask image corresponding to each reference object. Additionally, the electronic device can utilize a pre-trained object classification algorithm from related technologies to identify the object category of each reference object in the reference group photo image, obtaining a first object category identifier for each reference object. In this way, the object mask image and the first object category identifier for each reference object in the reference group photo image can be obtained.

[0086] S220. Extract the object description features of each target object from the portrait image, and perform object category recognition on the portrait image to determine the second object category identifier for each target object.

[0087] Specifically, the electronic device can extract features from a portrait image to obtain object description features for each target object. Furthermore, the electronic device can utilize pre-trained object classification algorithms from related technologies to identify the object category of each target object in the portrait image, obtaining a second object category identifier for each target object. In this way, object description features and a second object category identifier for each target object in the portrait image can be obtained.

[0088] S230. Based on the first object category identifier and the second object category identifier, determine the matching relationship between the target object and the reference object.

[0089] Specifically, when the user does not explicitly specify the matching relationship, the electronic device can use the matching relationship between the first object category identifier and the second object category identifier to determine the matching relationship between the target object and the reference object.

[0090] For example, when a portrait image contains two target objects of two different object categories (object category 1 and object category 2 in the example), and a reference group photo image also contains two reference objects of object category 1 and object category 2, a matching relationship can be established between the target object of object category 1 and the reference object of object category 1 according to the relationship of a perfect match between the first object category identifier and the second object category identifier, and at the same time, a matching relationship can be established between the target object of object category 2 and the reference object of object category 2.

[0091] For example, when a portrait image contains four target objects (two object categories 1 and two object categories 2), and a reference group photo image also contains four reference objects (two object categories 1 and two object categories 2), a matching relationship can be established between any target object of object category 1 and any reference object of object category 1, and between the remaining target object of the other object category 1 and the remaining reference object of the other object category 1, based on the premise that the first and second object category identifiers match, and combined with random matching rules. Simultaneously, a matching relationship can be established between any target object of object category 2 and any reference object of object category 2, and between the remaining target object of the other object category 2 and the remaining reference object of the other object category 2.

[0092] For example, when the first object category identifier corresponding to the portrait image and the second object category identifier corresponding to the reference group photo image cannot be matched well, a matching relationship between the target object and the reference object can be established randomly. For instance, when the portrait image contains three target objects of object category 1, and the reference group photo image contains three reference objects of object category 2, the target object and the reference object can be matched randomly.

[0093] S240. If the reference group photo image is determined through user input, then in response to the pairing operation between the target object and the reference object, the matching relationship is determined.

[0094] Specifically, when the reference group photo image is specified by the user, the user can see both the input image and the reference group photo image. In this case, the electronic device can provide an interactive function that matches the specified target object with the reference object.

[0095] For example, the above interactive function is implemented as an interactive operation of sequentially selecting objects for pairing. So, when the user triggers this interactive function and performs the interactive operation of sequentially selecting the target object in the portrait image and the reference object in the reference group photo image, the electronic device can establish the matching relationship between each target object and each reference object according to the user's selection order.

[0096] For example, the interactive functions described above implement interactive operations such as drop-down lists or selection options. The electronic device can then detect the user's selection and establish a matching relationship between the target object and the reference object based on the selection result.

[0097] S250. Based on the matching relationship, the object description features are fused into the object mask image to generate each initial fused image.

[0098] Specifically, according to the matching relationship described above, the object description features are fused into the corresponding object mask image to generate the fusion result of the target object (i.e., the initial fused image). In this initial fused image, the object mask region contains the fused object description features, while the pixel values ​​in other regions are 0.

[0099] S260. The initial fused image is fused with the reference group photo image to generate the initial group photo image.

[0100] Specifically, each initial fused image is then fused with a reference group photo image to retain the pixel values ​​of the object mask region in each initial fused image and the pixel values ​​of the region other than the object mask region in the reference group photo image, thus obtaining the initial group photo image.

[0101] It should be understood that the above-mentioned fusion process for generating the initial group photo image can perform certain transition processing on the boundary area between each object mask area and its adjacent non-object mask area to improve the smoothness of the initial group photo image.

[0102] The embodiments of this disclosure improve the matching accuracy between target objects and reference objects to a certain extent by introducing object category identifiers during the image fusion process, reduce the problem of unnatural fusion results caused by image fusion of different object categories, and further improve the naturalness and portrait similarity of the target group photo image.

[0103] Figure 3 This is provided by the embodiments of this disclosure. Figure 1 The diagram shows a refined flowchart of step S140 in an image generation method. (See attached diagram.) Figure 3 As shown, S140, "Based on the target diffusion model corresponding to the target object, adjust at least one object feature of the fusion object corresponding to the target object in the initial group photo image to generate a target group photo image," includes the following steps:

[0104] S310. Extract the initial local image containing the merged objects from the initial group photo image.

[0105] Specifically, the pre-trained candidate diffusion model is an image processing model for a specific single target object, and its processing effect on other target objects may be poor. Therefore, the electronic device first extracts images of the local regions where each fusion object is located from the initial group photo image, obtaining each initial local image. In this way, a candidate diffusion model can be applied only to its corresponding initial local image to adjust the object features of the adapted fusion object, which can greatly improve the image adjustment effect. Furthermore, by extracting local images, the amount of data computation can also be reduced to some extent, improving the efficiency of image adjustment.

[0106] S320. Based on the object identifier of the target object corresponding to the fusion object, select the target diffusion model that is suitable for the target object from multiple candidate diffusion models.

[0107] Specifically, according to the descriptions of the foregoing embodiments, in the embodiments of this disclosure, there exists a correspondence between a reference object, a fusion object, and a target object with the highest facial similarity to the fusion object, and a correspondence between the object identifier of the target object and a candidate diffusion model. Therefore, the electronic device can determine the corresponding target object based on the fusion object in the initial group photo image, and then determine a suitable candidate diffusion model for the fusion object based on the object identifier of the target object, which is then used as the target diffusion model.

[0108] In some embodiments, prior to S320, the method further includes: if the matching relationship between the target object and the reference object is random matching, then for any fusion object, determining the object similarity between the fusion object and each target object, and assigning the target object corresponding to the largest object similarity to the fusion object.

[0109] Specifically, the matching relationship between the target object and the reference object constructed in the aforementioned embodiments is used to fuse the target object into a suitable object mask image. However, due to various processing steps in the image fusion process, the portrait similarity between the resulting fused object and the target object may not be optimal. Therefore, directly selecting candidate diffusion models based on the above matching relationship may result in model mismatch, leading to a worse final image restoration effect. Therefore, before selecting candidate models, a correspondence between the fused object and the target object with the highest portrait similarity can be constructed first.

[0110] In cases where target and reference objects are randomly matched, the electronic device can calculate the object similarity between any fused object and each target object, and select the target object corresponding to the highest object similarity. Then, a correspondence is established between the fused object and the target object. Following this process, the best-matching target object for each fused object can be obtained, thereby improving the filtering accuracy of the diffusion model and further enhancing the portrait similarity and naturalness of the target group photo image.

[0111] In some other embodiments, prior to S320, the method further includes: if the matching relationship between the target object and the reference object is a one-to-one correspondence, then the target object corresponding to the reference object is matched with the fusion object.

[0112] Specifically, when the target object and the reference object are perfectly matched one-to-one, it reflects the user's fusion needs. Therefore, the matching relationship can be directly determined as the correspondence between the fusion object and the target object to further improve the fit between the target group photo image and the user's needs.

[0113] S330. Based on the image adjustment prompt and the initial local image corresponding to the target object, call the target diffusion model to adjust the image and generate the target local image.

[0114] Among them, the image adjustment prompt is a prompt used to trigger the running of the target diffusion model with image adjustment function. It can be a pre-built prompt or a prompt filled with information based on a pre-built prompt.

[0115] Specifically, after determining the target diffusion model corresponding to a certain fusion object, the electronic device can further determine the image adjustment prompt words that it is suitable for, and input the image adjustment prompt words and the initial local image corresponding to the fusion object into the target diffusion model. After the model calculation, the object adjustment result of the initial local image is output, that is, the target local image.

[0116] In some embodiments, prior to S330, the method further includes: constructing an image adjustment prompt based on a first object category identifier and a first preset prompt word of the reference object corresponding to the fused object.

[0117] The first preset prompt is a pre-built prompt word for the model applied to the image adjustment function. It contains some pre-designed text content, such as placeholders for object category identification, image adjustment process, output format, etc.

[0118] Specifically, given that the final group photo image primarily reflects information about the reference objects, the electronic device can limit the adjustment effect on the merged objects based on the first object category identifier of the reference objects. Therefore, the electronic device can fill the object category identifier placeholder in the first preset prompt word with the first object category identifier of the reference object to obtain the image adjustment prompt word corresponding to that reference object. In this way, when the image adjustment model refers to this image adjustment prompt word to adjust the image, it can further improve the object fusion effect. For example, the adjustment result of merged objects in the urban category is more sophisticated, and the adjustment result of merged objects in the leisure category is more sunny, thereby further enhancing the naturalness of the target group photo image.

[0119] S340. Fuse the local image of the target into the initial group photo image to generate the target group photo image.

[0120] Specifically, the electronic device fuses the local images of each target back into the corresponding image region of the initial group photo image to obtain the target group photo image.

[0121] The embodiments of this disclosure apply a target diffusion model adapted to the target object to the initial local image corresponding to each fusion object. This allows for targeted adjustment of at least one object feature of each fusion object, thereby reducing data processing volume, improving image adjustment efficiency, and enhancing the portrait similarity between the corresponding fusion object and the target object in the target group photo image, as well as the naturalness of the final fused portrait.

[0122] Based on the above embodiments, the process of generating personalized target group photos will be explained using examples such as portrait image 1 containing a person of a single object category 1, portrait image 2 containing a person of a single object category 2, and a reference group photo image containing two people, one from object category 1 and the other from object category 2. See [link to documentation]. Figure 4 The specific implementation process of this image generation method is as follows:

[0123] The user inputs portrait image 1, portrait image 2, and a reference group photo image. ① Object category recognition and mask segmentation are performed on the reference group photo image to obtain object mask image 1 carrying object category 1 identifiers and object mask image 2 carrying object category 2 identifiers. ② The matching relationship between the target objects in object mask image 1 and portrait image 1, and the matching relationship between the target objects in object mask image 2 and portrait image 2 are established. Then, the target objects in portrait image 1 are fused to object mask image 1 to obtain initial fused image 1, and the target objects in portrait image 2 are fused to object mask image 2 to obtain initial fused image 2. Afterwards, initial fused image 1, initial fused image 2, and the reference group photo image are fused to obtain the initial group photo image. From the initial group photo image, it can be seen that there are obvious fusion edges in the head areas (such as the jawline and hairline) of the two target objects, and the texture and details of their facial areas are relatively coarse, exhibiting a noticeable mask-like appearance. ③ Initial local images corresponding to the two figures are extracted from the initial group photo image. Then, using the object category identifiers, image adjustment prompts corresponding to object category 1 and object category 2 are obtained respectively. Furthermore, based on the correspondence between the fusion object and the target object, target diffusion model 1 for people in object category 1 and target diffusion model 2 for people in object category 2 are obtained respectively. Next, using the image adjustment prompts corresponding to object category 1 and target diffusion model 1, the initial local image corresponding to the people in object category 1 is adjusted to obtain target local image 1. Then, using the image adjustment prompts corresponding to object category 2 and target diffusion model 2, the initial local image corresponding to the people in object category 2 is adjusted to obtain target local image 2. Finally, target local image 1, target local image 2, and the initial group photo image are fused to obtain the target group photo image.

[0124] The following are embodiments of the image generation apparatus provided in this invention. This apparatus and the image generation methods of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the image generation apparatus, please refer to the embodiments of the above image generation methods.

[0125] Figure 5 A schematic diagram of the structure of an image generation apparatus provided in an embodiment of this disclosure is shown. For example... Figure 5 As shown, the image generation apparatus 500 may include:

[0126] The portrait image acquisition module 510 is used to acquire at least one portrait image in response to a target interaction operation; wherein the portrait image contains one or more target objects to be processed;

[0127] The reference group photo image determination module 520 is used to determine a reference group photo image; wherein the number of reference objects in the reference group photo image is the same as the number of target objects;

[0128] The initial group photo image generation module 530 is used to generate an initial group photo image by fusing the target object into the reference object in the reference group photo image based on the portrait image;

[0129] The target group photo image generation module 540 is used to adjust at least one object feature of the fusion object corresponding to the target object in the initial group photo image based on the target diffusion model corresponding to the target object, so as to generate the target group photo image; wherein, the target diffusion model is obtained by fine-tuning and training the initial diffusion model based on the image samples of the target object.

[0130] The image generation apparatus provided in this disclosure can acquire portrait images containing multiple target objects to be processed and determine reference group photos containing the same number of reference objects. Then, each target object is initially fused into the reference group photo image to obtain an initial group photo image containing multiple fused objects. Then, using a target diffusion model adapted to each target object and finely trained based on image samples of the target object, at least one object feature of the fused object corresponding to the local target object in the initial group photo image is adjusted to greatly reduce the image problems of insufficient similarity and unnatural visual effect of the fused objects in the initial group photo image, thereby obtaining the final target group photo image. On the one hand, it can generate personalized group photo images by inputting portrait images of unrestricted target objects, improving the flexibility and diversity of image generation. On the other hand, it improves the similarity between each object in the target group photo image and the corresponding target object, and enhances the naturalness of each object in the target group photo image, thereby improving the realism and image quality of personalized image generation.

[0131] In some embodiments, the initial group photo image generation module 530 includes:

[0132] The first object feature generation submodule is used to generate the first object feature of each reference object based on the reference group photo image;

[0133] The second object feature extraction submodule is used to extract the second object features of each target object from the portrait image;

[0134] The initial group photo image generation submodule is used to generate an initial group photo image by fusing the target object into a reference object in a reference group photo image based on the first object features and the second object features.

[0135] In some embodiments, the first object feature includes an object mask image and a first object category identifier; the second object feature includes an object description feature and a second object category identifier.

[0136] Accordingly, the initial group photo image generation submodule is specifically used for:

[0137] Based on the first object category identifier and the second object category identifier, determine the matching relationship between the target object and the reference object;

[0138] Based on the matching relationship, the object description features are fused into the object mask image to generate an initial fused image;

[0139] The initial fused image is merged with the reference group photo image to generate the initial group photo image.

[0140] In some embodiments, the initial group photo image generation submodule is further configured to:

[0141] Before fusing the descriptive features of each object into the mask image of each object based on the matching relationship to generate each initial fused image, if the reference group photo image is determined through user input, the matching relationship is determined in response to the pairing operation of the target object and the reference object.

[0142] In some embodiments, the target group photo image generation module 540 includes:

[0143] The initial local image extraction submodule is used to extract the initial local image containing the fused objects from the initial group photo image;

[0144] The target diffusion model filtering submodule is used to filter the target diffusion model that is suitable for the target object from multiple candidate diffusion models based on the object identifier of the target object corresponding to the fusion object.

[0145] The target local image generation submodule is used to adjust the initial local image corresponding to the image prompt and the target object based on the image adjustment prompt and the target diffusion model to generate the target local image;

[0146] The target composite image generation submodule is used to fuse the local image of the target into the initial composite image to generate the target composite image.

[0147] In some embodiments, the target group photo image generation module 540 further includes an image restoration prompt word construction submodule, used for:

[0148] Before generating the target local image by calling the target diffusion model to adjust the image based on the image adjustment prompt words and the initial local image corresponding to the target object, the image adjustment prompt words are constructed based on the first object category identifier and the first preset prompt words of the reference object corresponding to the fusion object.

[0149] In some embodiments, the target group photo image generation module 540 further includes an object correspondence submodule, used for:

[0150] Before selecting a target diffusion model that matches the target object from multiple candidate diffusion models based on the object identifier of the target object corresponding to the fusion object, if the matching relationship between the target object and the reference object is random matching, then for any fusion object, the object similarity between the fusion object and each target object is determined, and the target object corresponding to the largest object similarity is assigned to the fusion object.

[0151] If the matching relationship between the target object and the reference object is a one-to-one correspondence, then the target object corresponding to the reference object will be matched with the fusion object.

[0152] In some embodiments, the reference group photo image determination module 520 is specifically used for:

[0153] In response to the image upload operation, the received group photo image is determined as the reference group photo image;

[0154] Alternatively, in response to the first image selection operation, the selected candidate group photo image is determined as the reference group photo image.

[0155] In other embodiments, the reference group photo image determination module 520 is specifically used for:

[0156] Identify the target text containing a description of the group photo;

[0157] Based on the target text and the second preset prompt, an image generation prompt is constructed, and an image generation model is called based on the image generation prompt to generate a reference group photo image.

[0158] In some embodiments, the reference group photo image determination module 520 is further specifically configured to determine target text containing a group photo description by any of the following:

[0159] In response to a text input operation, the received text is identified as the target text;

[0160] Generate target text based on the object information of the target object;

[0161] Target text is generated based on the object information of the target object and the style of the target group photo.

[0162] In some other embodiments, the reference group photo image determination module 520 is specifically used for:

[0163] Based on the object information of the target object, a reference group photo image is selected from multiple candidate group photo images;

[0164] Alternatively, reference group photos can be selected from multiple candidate group photos based on the object information of the target object and the style of the target group photo.

[0165] In some embodiments, the portrait image acquisition module 510 is specifically used for:

[0166] In response to the second image selection operation, the selected image in the image library is determined to be a portrait image; wherein the selected image contains one or more target objects;

[0167] Alternatively, in response to an image acquisition operation, the acquired image is determined to be a portrait image; wherein the acquired image contains one or more target objects.

[0168] The image generation apparatus provided in the embodiments of the present invention can execute the image generation method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0169] It is worth noting that in the embodiments of the above image generation device, the various modules and sub-modules are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional module / sub-module are only for easy differentiation and are not used to limit the protection scope of this disclosure.

[0170] This disclosure also provides an electronic device that may include a processor and a memory, the memory being used to store executable instructions. The processor can be used to read the executable instructions from the memory and execute the executable instructions to implement the image generation method described above.

[0171] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this disclosure is shown.

[0172] like Figure 6As shown, the electronic device 600 may include a processing unit 601 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. The RAM 603 also stores various programs and data required for the operation of the electronic device 600. The processing unit 601, ROM 602, and RAM 603 are interconnected via a bus 604. An input / output interface (I / O interface) 605 is also connected to the bus 604.

[0173] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touch screens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data.

[0174] It should be noted that, Figure 6 The illustrated electronic device 600 is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein. That is, although... Figure 6 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0175] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the image generation method of any embodiment of this disclosure.

[0176] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the image generation method in any embodiment of this disclosure.

[0177] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0178] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP, and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0179] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0180] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the image generation method described in any embodiment of this disclosure.

[0181] In embodiments of this disclosure, computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include, but are not limited to, object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0182] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0183] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0184] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0185] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0186] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0187] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An image generation method characterized by, The method comprises: in response to a target interaction operation, obtaining at least one portrait image; wherein the portrait image contains one or more target objects to be processed; determining a reference group photo image; wherein the number of reference objects in the reference group photo image is the same as the number of target objects; based on the portrait image, fusing the target objects to the reference objects in the reference group photo image to generate an initial group photo image; based on a target diffusion model corresponding to the target object, adjusting at least one object feature of a fusion object corresponding to the target object in the initial group photo image to generate a target group photo image; wherein the target diffusion model is obtained by fine-tuning training an initial diffusion model based on image samples of the target object.

2. The method of claim 1, wherein, The method comprises: based on the reference group photo image, generating a first object feature of each reference object; extracting a second object feature of each target object from the portrait image; based on the first object feature and the second object feature, fusing the target objects to the reference objects in the reference group photo image to generate the initial group photo image.

3. The method of claim 2, wherein, The first object feature comprises an object mask image and a first object category identifier; the second object feature comprises an object description feature and a second object category identifier. The method comprises: based on the first object category identifier and the second object category identifier, determining a matching relationship between the target objects and the reference objects; based on the matching relationship, fusing the object description feature to the object mask image to generate an initial fusion image; fusing the initial fusion image with the reference group photo image to generate the initial group photo image.

4. The method of claim 3, wherein, Before the method of fusing the object description feature to the object mask image based on the matching relationship to generate an initial fusion image, the method further comprises: if the reference group photo image is determined by user input operation, then in response to a pairing operation of the target objects and the reference objects, determining the matching relationship.

5. The method of claim 1, wherein, The method comprises: extracting an initial local image where the fusion object is located from the initial group photo image; based on the object identifier of the target object corresponding to the fusion object, screening the target diffusion model that fits the target object from a plurality of candidate diffusion models; based on an image adjustment prompt word and the initial local image corresponding to the target object, calling the target diffusion model to perform image adjustment to generate a target local image; fusing the target local image to the initial group photo image to generate the target group photo image.

6. The method of claim 5, wherein, Before the image adjustment prompt word is constructed based on the initial local image of the target object corresponding to the fusion object, and the target diffusion model is called for image adjustment to generate a target local image, the method further includes: Based on the first object class identifier and the first preset prompt word of the reference object corresponding to the fusion object, the image adjustment prompt word is constructed.

7. The method of claim 5, wherein, Before the target diffusion model that adapts to the target object is selected from a plurality of candidate diffusion models based on the object identifier of the target object corresponding to the fusion object, the method further includes: If the matching relationship between the target object and the reference object is random matching, for any fusion object, the object similarity between the fusion object and each target object is determined, and the target object corresponding to the maximum object similarity corresponds to the fusion object. If the matching relationship between the target object and the reference object is one-to-one correspondence matching, the target object corresponding to the reference object corresponds to the fusion object.

8. The method of claim 1, wherein, The reference group photo image is determined, including: In response to an image upload operation, the received group photo image is determined as the reference group photo image; Or, in response to a first image selection operation, the selected candidate group photo image is determined as the reference group photo image.

9. The method of claim 1, wherein, The reference group photo image is determined, including: Determine the target text containing the group photo description; Based on the target text and the second preset prompt word, an image generation prompt word is constructed, and an image generation model is called based on the image generation prompt word to generate the reference group photo image.

10. The method of claim 9, wherein, The target text containing the group photo description is determined, including any of the following: In response to a text input operation, the received text is determined as the target text; Based on the object information of the target object, the target text is generated; Based on the object information of the target object and the target group photo style, the target text is generated.

11. The method of claim 1, wherein, The reference group photo image is determined, including: Based on the object information of the target object, the reference group photo image is selected from a plurality of candidate group photo images; Or, based on the object information of the target object and the target group photo style, the reference group photo image is selected from a plurality of candidate group photo images.

12. The method of claim 1, wherein, In response to a target interaction operation, at least one portrait image is acquired, including: In response to a second image selection operation, the selected image in the image library is determined as the portrait image; wherein the selected image contains one or more target objects; Or, in response to an image acquisition operation, the acquired image is determined as the portrait image; wherein the acquired image contains one or more target objects.

13. An image generation apparatus characterized by comprising: It includes: A portrait image acquisition module is configured to acquire at least one portrait image in response to a target interaction operation; wherein the portrait image contains one or more target objects to be processed; A reference group photo image determination module is configured to determine a reference group photo image; wherein the number of reference objects in the reference group photo image is the same as the number of target objects; A reference group photo image determination module is configured to determine a reference group photo image; wherein the number of reference objects in the reference group photo image is the same as the number of target objects; An initial group photo image generation module is configured to fuse the target object into the reference object in the reference group photo image based on the portrait image, and generate an initial group photo image; A target group photo image generation module is configured to adjust at least one object feature of a fused object corresponding to the target object in the initial group photo image based on a target diffusion model corresponding to the target object, and generate a target group photo image; the target diffusion model is obtained by fine-tuning training of an initial diffusion model based on image samples of the target object.

14. An electronic device, comprising: Comprise: A processor; A memory for storing executable instructions; The processor is configured to read the executable instructions from the memory and execute the executable instructions to implement the image generation method of any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and when the computer program is executed by the processor, the processor implements the image generation method of any one of claims 1 to 12. The storage medium stores a computer program, and when the computer program is executed by the processor, the processor implements the image generation method of any one of claims 1 to 12.

Images