Image generation method and device, intelligent agent, intelligent agent system and storage medium

Abstract

The invention provides an image generation method and device, an intelligent agent, an intelligent agent system and a storage medium, relates to the field of artificial intelligence, in particular to the fields of computer vision, deep learning and large models, and can be applied to content generation scenes based on artificial intelligence. The specific implementation scheme is as follows: acquiring image generation demand information; determining a corresponding target image generation mode according to the image generation demand information; querying to obtain a first reference image based on the image generation demand information; and based on the image generation demand information and the first reference image, adopting a target image generation mode to generate a target image.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically computer vision, deep learning, large models and other technical fields, and can be applied to artificial intelligence-based content generation (AIGC) scenarios, and in particular to an image generation method, device, intelligent agent, intelligent agent system and storage medium. Background Art

[0002] The core of AI image generation technology based on AIGC is to use AI image generation models to achieve text-to-image or image-to-image conversion. However, the image generation effect of AI image generation technology is highly dependent on the quality and timeliness of model training data. Specifically, after the AI ​​image generation model is trained, the image content it generates is often limited by the timeliness of the model training data. Before the model training data is updated, the AI ​​image generation model cannot obtain the latest content, and the image it generates may have a certain lag. Summary of the invention

[0003] The present disclosure provides an image generation method, device, intelligent agent, intelligent agent system and storage medium.

[0004] According to a first aspect of the present disclosure, there is provided an image generation method, comprising: obtaining image generation requirement information; determining a corresponding target image generation method according to the image generation requirement information; obtaining a first reference image by querying based on the image generation requirement information; and generating a target image by adopting the target image generation method based on the image generation requirement information and the first reference image.

[0005] According to a second aspect of the present disclosure, there is provided an image generation method, including: acquiring image generation requirement information; determining a corresponding target image generation method according to the image generation requirement information; judging whether a reference image query is required for the image generation requirement information; in a case where a reference image query is required for the image generation requirement information, querying and obtaining a first reference image based on the image generation requirement information; and generating a target image using the target image generation method based on the image generation requirement information and the first reference image.

[0006] According to a third aspect of the present disclosure, there is provided an image generating device, comprising: an acquisition module for acquiring image generation requirement information; a determination module for determining a corresponding target image generating method according to the image generation requirement information; a query module for querying and obtaining a first reference image based on the image generation requirement information; and a generation module for generating a target image using the target image generating method based on the image generation requirement information and the first reference image.

[0007] According to a fourth aspect of the present disclosure, an image generating device is provided, comprising: an acquisition module for acquiring image generation requirement information; a determination module for determining a corresponding target image generating method according to the image generation requirement information; a judgment module for judging whether a reference image query is required for the image generation requirement information; a query module for querying and obtaining a first reference image based on the image generation requirement information when a reference image query is required for the image generation requirement information; and a generation module for generating a target image by adopting the target image generating method based on the image generation requirement information and the first reference image.

[0008] According to a fifth aspect of the present disclosure, an intelligent agent is provided, comprising: an input module for acquiring image generation requirement information; a processing module for determining a corresponding target image generation method according to the image generation requirement information; querying and obtaining a first reference image based on the image generation requirement information; generating a target image based on the image generation requirement information and the first reference image using the target image generation method; and an output module for outputting the target image.

[0009] According to a sixth aspect of the present disclosure, an intelligent agent is provided, comprising: an input module for acquiring image generation requirement information; a processing module for determining a corresponding target image generation method according to the image generation requirement information; judging whether a reference image query is required for the image generation requirement information; in a case where a reference image query is required for the image generation requirement information, querying and obtaining a first reference image based on the image generation requirement information; generating a target image based on the image generation requirement information and the first reference image using the target image generation method; and an output module for outputting the target image.

[0010] According to a seventh aspect of the present disclosure, there is provided an intelligent agent system, comprising:

[0011] at least one processor; and

[0012] a memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the image generation method described in the first aspect or the image generation method described in the second aspect.

[0014] According to an eighth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the image generation method described in the first aspect or the image generation method described in the second aspect.

[0015] According to a ninth aspect of the present disclosure, a computer program product is provided, comprising computer instructions, which, when executed by a processor, implement the steps of the image generation method described in the first aspect or the steps of the image generation method described in the second aspect.

[0016] The present disclosure provides an image generation method, device, agent, agent system and storage medium, which have the following beneficial effects:

[0017] Obtain image generation requirement information; determine the corresponding target image generation method according to the image generation requirement information; query and obtain a first reference image based on the image generation requirement information; generate a target image using a target image generation method based on the image generation requirement information and the first reference image. The present disclosure generates a target image based on the queried first reference image, that is, it adopts image retrieval-augmented generation (iRAG) technology, and the timeliness of image generation is aligned with the timeliness of image search, avoiding the problem of possible lag in the target image. In addition, since image search covers almost all public knowledge, the present disclosure has no memory capacity bottleneck, and also solves the problem of limited memory capacity of AI image generation models.

[0018] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0020] Figure 1 is a flowchart of an image generation method provided according to an embodiment of the present disclosure;

[0021] Figure 2 is a flowchart of an image generating method provided according to another embodiment of the present disclosure;

[0022] Figure 3 is a flowchart of an image generating method provided according to another embodiment of the present disclosure;

[0023] Figure 4 is a schematic diagram of a text-image interlacing feature provided according to an embodiment of the present disclosure;

[0024] Figure 5 is a flowchart of an image generating method provided according to another embodiment of the present disclosure;

[0025] Figure 6 is a flowchart of an image generation method provided according to an embodiment of the present disclosure;

[0026] Figure 7 is a flowchart of an image generating method provided according to another embodiment of the present disclosure;

[0027] Figure 8 A schematic diagram of the structure of an image generating device provided according to an embodiment of the present disclosure;

[0028] Fig. 9 A schematic diagram of the structure of an image generating device provided according to an embodiment of the present disclosure;

[0029] Fig.10 A schematic diagram of an intelligent agent provided according to an embodiment of the present disclosure;

[0030] Fig.11 It is a block diagram of an intelligent agent system used to implement the image generation method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0032] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, comply with the relevant laws and regulations, and do not violate public order and good morals.

[0033] The following describes the image generation method, device, agent, agent system and storage medium of the embodiments of the present disclosure with reference to the accompanying drawings.

[0034] It should be noted that the execution subject of the image generation method of this embodiment is an image generation device, which can be implemented by software and / or hardware and can be configured in an intelligent agent.

[0035] Figure 1 It is a flowchart of an image generating method provided according to an embodiment of the present disclosure.

[0036] like Figure 1As shown, the image generation method includes:

[0037] Step 101, obtaining image generation requirement information.

[0038] The image generation requirement information may be used to indicate information such as the image style, image subject, and image size of the image to be generated.

[0039] As an example, the image generation requirement information may include an image generation requirement prompt input by a user.

[0040] As another example, the image generation requirement information may include a prompt input by the user and a reference image.

[0041] Step 102: determining a corresponding target image generation method according to image generation requirement information.

[0042] The image generation method corresponds to the image generation requirements. Different image generation requirements require different image generation methods. For example, different image generation methods may correspond to different AI image generation models.

[0043] As an example, the target image generation mode corresponding to the image generation requirement information may be determined according to the correspondence between the image generation requirement and the image generation mode.

[0044] As an example, a target image generation method corresponding to the image generation requirement information may be determined based on a large model.

[0045] Step 103: Based on the image generation requirement information, query and obtain a first reference image.

[0046] Wherein, based on the image generation requirement information, an image query can be performed in a set image library to obtain a first reference image. It should be noted that the first reference image can be one or more images.

[0047] As an example, vector features corresponding to the image generation requirement information may be obtained, and similarity matching may be performed between the vector features and description features of each image in the image library to obtain a first reference image.

[0048] Step 104 : generating a target image by adopting a target image generating method based on the image generating requirement information and the first reference image.

[0049] The image generation requirement information and the first reference image are inputs of the target image generation method. After obtaining the image generation requirement information and the first reference image, the image generation requirement information and the first reference image can be processed according to the target image generation method to obtain the target image.

[0050] It should be noted that, when the image generation requirement information includes a reference image, the user's editing intention for the reference image can be obtained based on the image generation requirement information; based on the image generation requirement information, the first reference image and the editing intention, a target image generation method is adopted to generate the target image.

[0051] In the embodiment of the present disclosure, image generation requirement information is obtained; the corresponding target image generation method is determined according to the image generation requirement information; based on the image generation requirement information, a first reference image is queried; based on the image generation requirement information and the first reference image, a target image generation method is adopted to generate a target image. The present disclosure generates a target image based on the queried first reference image, that is, image retrieval-augmented generation (iRAG) technology is adopted, and the timeliness of image generation is aligned with the timeliness of image search, avoiding the problem of possible lag of the target image. In addition, since image search covers almost all public knowledge, the present disclosure has no memory capacity bottleneck, and also solves the problem of limited memory capacity of AI image generation models.

[0052] Figure 2 It is a flowchart of an image generating method provided according to another embodiment of the present disclosure.

[0053] like Figure 2 As shown, the image generation method includes:

[0054] Step 201, obtaining image generation requirement information.

[0055] Step 202: Determine a corresponding target image generation method according to the image generation requirement information.

[0056] Step 203, obtaining the image subject information included in the image generation requirement information; based on the image subject information, performing an image search in a set image library to obtain a first reference image.

[0057] The image generation requirement information includes image subject information, and the image subject information may refer to subject type, subject name and other information.

[0058] For example, assuming that the image generation requirement information is "draw a red vehicle with a stylish and modern design and a futuristic atmosphere", the main image information may be a red vehicle. It should be noted that if the image generation requirement information also includes a vehicle model, the main image information may be a red vehicle and a vehicle model.

[0059] As an example, the image main body information in the image generation requirement information can be extracted based on the large model, and then an image search is performed based on the image main body information to obtain the first reference image.

[0060] By performing image queries based on the main information of the image, reference images that match the image generation requirements can be accurately screened out in the image library, thereby improving the image generation effect.

[0061] The quality of the first reference image will affect the image generation effect. If the quality of the first reference image is poor, the quality of the final generated image may also be poor. Therefore, in order to ensure the image generation effect and further improve the quality of the generated image, as an example, based on the image subject information, an image query is performed in a set image library to obtain a candidate reference image; the image quality of the candidate reference image is obtained, and the set quality requirement corresponding to the image generation requirement information is obtained; the candidate reference images are screened according to the image quality and the set quality requirement to obtain the first reference image.

[0062] Different image generation requirements have different corresponding quality requirements. For example, if the image generation requirement is to generate a portrait, the corresponding quality requirement may be that the facial clarity in the image must be higher than the set clarity threshold; if the image generation requirement is to generate a poster image, the corresponding quality requirement may be that the image does not include a watermark.

[0063] As a possible implementation manner, it is possible to determine whether the image quality of the candidate reference image meets the set quality requirement and use the candidate reference image whose image quality meets the set quality requirement as the first reference image.

[0064] As another possible implementation, the candidate reference images may be sorted in descending order of image quality according to the image quality and the set quality requirement, and the first N candidate reference images may be used as the first reference images; wherein N is a positive integer.

[0065] Step 204 : Generate a target image by adopting a target image generation method based on the image generation requirement information and the first reference image.

[0066] It should be noted that the explanation of step 201, step 202 and step 204 can be found in the relevant description in any embodiment of the present disclosure, and will not be repeated here.

[0067] In the disclosed embodiment, image generation requirement information is obtained; a corresponding target image generation method is determined according to the image generation requirement information; image subject information included in the image generation requirement information is obtained; based on the image subject information, an image query is performed in a set image library to obtain a first reference image; based on the image generation requirement information and the first reference image, a target image is generated using a target image generation method. By performing an image query based on the image subject information, a reference image matching the image generation requirement can be accurately screened out in the image library, thereby improving the image generation effect.

[0068] Figure 3 It is a flowchart of an image generating method provided according to another embodiment of the present disclosure.

[0069] like Figure 3 As shown, the image generation method includes:

[0070] Step 301, obtaining image generation requirement information.

[0071] Step 302: Obtain the requirement text in the image generation requirement information.

[0072] The requirement text may refer to a prompt in the image generation requirement information.

[0073] Step 303, input the requirement text into the first large model to perform subject modification intention detection, and obtain the target subject modification intention corresponding to the image generation requirement information, wherein the target subject modification intention is used to indicate whether the subject in the first reference image needs to be modified.

[0074] Among them, the first large model can determine whether the subject in the first reference image needs to be modified when generating the image based on the requirement text. If the target subject modification intention indicates that the subject in the first reference image needs to be modified, the image generation requirement information corresponds to a generalized requirement scenario under high cognition; if the target subject modification intention indicates that the subject in the first reference image does not need to be modified, the image generation requirement information corresponds to a pixel-level fidelity requirement scenario.

[0075] For example, if the image generation requirement information is "generate an image of a model A vehicle driving in a desert" or "generate an image of a model A vehicle traveling through a forest", the target subject modification intention corresponding to the image generation requirement information is that the subject in the first reference image does not need to be modified. If the image generation requirement information is "generate an image of a person B wearing clothing of style C" or "generate an image of a doll of person B", the target subject modification intention corresponding to the image generation requirement information is that the subject in the first reference image needs to be modified.

[0076] It should be noted that, if the image generation requirement information includes a second reference image input by the user, the subject modification intention obtained by subject modification intention detection can also be used to indicate whether the subject in the second reference image needs to be modified.

[0077] Step 304 : based on the mapping relationship between the subject modification intention and the image generation method, obtain the target image generation method corresponding to the target subject modification intention.

[0078] Different subject modification intentions correspond to different image generation methods. As an example, a mapping relationship between subject modification intentions and image generation methods can be preset, and after determining the target subject modification intention, the corresponding target image generation method can be determined based on the preset mapping relationship.

[0079] Different subject modification intentions require different focuses when generating images. Therefore, selecting an image generation method that matches the subject modification intention can significantly improve the effect and efficiency of image generation.

[0080] Step 305: Based on the image generation requirement information, query and obtain a first reference image.

[0081] Step 306: Generate a target image by adopting a target image generation method based on the image generation requirement information and the first reference image.

[0082] In order to extract key features from the requirement text, the requirement text can be encoded to obtain text features.

[0083] The following describes the image generation process by combining the two image generation methods corresponding to the subject modification intentions:

[0084] As an example, in response to the target subject modification intention indicating that the subject in the first reference image does not need to be modified, feature extraction is performed on the first reference image to obtain first image features, and a subject segmentation image in the first reference image is obtained; the first image features and text features are input into the first raw image model to obtain the background image and subject layout information; based on the subject layout information, the subject segmentation image and the background image are fused to obtain the target image.

[0085] Among them, the image generation method corresponding to this example can be called an accurate image generation method; the first image feature can refer to a vector feature obtained by using a small-scale feature extractor; the subject segmentation image can refer to an image of the subject part obtained by semantically segmenting the first reference image; the background image can refer to an image of the background part other than the subject part; the subject layout information is used to indicate the layout of the subject in the background image. For example, the subject layout information includes information such as the position and size of the subject in the background image.

[0086] The present disclosure obtains the content of the first reference image from the pixel level, namely the subject segmentation image. The subject segmentation image not only retains the key details and features of the subject in the first reference image, achieves strong fidelity of the subject, but also ensures the integrity of the subject in the target image. In addition, image fusion based on subject layout information can make the subject naturally integrated into the background image and maintain harmony and unity with other parts of the background image.

[0087] As another example, in response to the target subject modification intention indicating that the subject in the first reference image needs to be modified, feature extraction is performed on the first reference image to obtain second image features; based on the description objects corresponding to the text features and each sub-feature in the second image features, the text features and the second image features are interlaced and spliced ​​to obtain text-image interlaced features; the text-image interlaced features are input into the second raw image model to obtain the target image.

[0088] Among them, the image generation method corresponding to this example can be called a highly generalized image generation method; the second image feature is a vector feature, and the second image feature and the first image feature may be different; the text feature includes text sub-features, and the image feature includes image sub-features, and the text sub-features and image sub-features correspond to description objects respectively; interlaced splicing refers to combining image features and text features in a certain way (for example, the text sub-features and image sub-features corresponding to the same description object are spliced ​​together) to form a new coding feature, that is, the image-text interlaced feature, please refer to Figure 4 , Figure 4 This is a schematic diagram of the interlaced features of images and texts. This method combines information of two different modalities, image features and text features, in a specific way to achieve the association between multimodal features.

[0089] For example, assuming that the image generation requirement information is "generate action figure doll images of character B and character D", the corresponding description objects include character B and character D. Character B has corresponding text sub-features and image sub-features, and character D also has corresponding text sub-features and image sub-features. When performing interlaced splicing, the image sub-features corresponding to character B can be inserted after the text sub-features corresponding to character B in the text features, and the image sub-features corresponding to character D can be inserted after the text sub-features corresponding to character D in the text features.

[0090] The text features and the second image features are interlaced and spliced ​​to achieve precise alignment of the two in describing the object. This interlaced splicing method not only retains the key information in each feature, but also promotes the deep fusion between text features and image features. When generating the target image based on the interlaced features of text and images, it can not only fully consider the characteristics of the subject in the first reference image, but also perform generalized secondary creation while taking into account the fidelity of the subject, and the image generation effect is better.

[0091] The image generation requirement information may include a second reference image input by the user. In this case, the method for obtaining the interlaced image and text features is as follows: extract features from the second reference image to obtain third image features; based on the description objects corresponding to the text features, the second image features and the sub-features in the third image features, interlace and splice the text features, the second image features and the third image features to obtain the interlaced image and text features.

[0092] For example, assuming that the image generation requirement information includes a second reference image of character B and "generate a group photo image of action figures of character B and character D", when generating the target image, it is necessary to interlace and splice the text sub-features corresponding to character B and the image sub-features of character B in the third image features, and to interlace and splice the text sub-features corresponding to character D and the image sub-features of character D in the second image features.

[0093] When the image generation requirement information includes a second reference image input by the user, the features are interlaced and spliced ​​based on the requirement text, the first reference image and the second reference image input by the user to generate a target image. This can not only meet the user's personalized image generation requirements and achieve flexible personalized image generation, but also ensure the image generation effect.

[0094] In addition, for non-public knowledge, the present invention can avoid the problem of raw image models lacking memory of these contents by allowing users to independently provide reference images.

[0095] It should be noted that the explanation of step 301, step 302 and step 305 can be found in the relevant description in any embodiment of the present disclosure, and will not be repeated here.

[0096] In the disclosed embodiment, image generation requirement information is obtained; requirement text in the image generation requirement information is obtained; the requirement text is input into the first large model to detect subject modification intention, and the target subject modification intention corresponding to the image generation requirement information is obtained, wherein the target subject modification intention is used to indicate whether the subject in the first reference image needs to be modified; based on the mapping relationship between the subject modification intention and the image generation method, the target image generation method corresponding to the target subject modification intention is obtained; based on the image generation requirement information, the first reference image is obtained by query; based on the image generation requirement information and the first reference image, the target image is generated by the target image generation method. Different subject modification intentions require different focuses when generating images. Therefore, selecting an image generation method that matches the subject modification intention can significantly improve the effect and efficiency of image generation.

[0097] Figure 5 It is a flowchart of an image generating method provided according to another embodiment of the present disclosure.

[0098] like Figure 5 As shown, the image generation method includes:

[0099] Step 501: Obtain image generation requirement information.

[0100] Step 502: Determine a corresponding target image generation method according to the image generation requirement information.

[0101] Step 503: Based on the image generation requirement information, query and obtain a first reference image.

[0102] Step 504, obtaining the requirement text in the image generation requirement information; rewriting and expanding the requirement text to obtain the target requirement text; and generating the target image based on the target requirement text and the first reference image by adopting the target image generation method.

[0103] Among them, compared with the demand text, the target demand text has a more regular format and a more comprehensive detailed description. As an example, the demand text can be input into the big model, so that the big model can rewrite and expand the demand text based on the core picture content, picture style description, picture subject and limitation description, picture detail description, picture background modification description, special effects, composition, color tone, clarity description, quality description and other dimensions.

[0104] The requirement text in the image generation requirement information may have problems such as abstractness and vague expression. Directly generating images based on the requirement text often cannot produce the best output results. Therefore, the present invention rewrites and expands the requirement text to improve the image generation effect.

[0105] As another example, context information of image generation requirement information is obtained; when the image generation requirement information includes a second reference image input by the user, description information of the second reference image is obtained; at least one item of the context information and the description information, as well as the requirement text, is input into the second largest model for rewriting and expansion to obtain the target requirement text.

[0106] The second largest model and the first largest model may be the same large model or may refer to different large models.

[0107] During the image generation process, the image generation requirement information provided by the user may sometimes be incomplete, which may cause the generated image to deviate from the user's expectations. In order to more accurately understand the user's needs and improve the accuracy and effect of image generation, the present disclosure can further obtain context information related to the image generation requirement information, and obtain the description information of the second reference image when the user inputs the second reference image. Combined with this information, the present disclosure can more comprehensively understand the user's true image generation intention, thereby improving the image generation effect and improving the user experience.

[0108] It should be noted that the explanation of steps 501 to 503 can be found in the relevant description in any embodiment of the present disclosure and will not be repeated here.

[0109] In the embodiment of the present disclosure, the requirement text in the image generation requirement information is obtained; the requirement text is rewritten and expanded to obtain the target requirement text; based on the target requirement text and the first reference image, the target image is generated using the target image generation method. The requirement text in the image generation requirement information may be abstract and vague in expression, and the best output result is often not obtained by directly generating an image based on the requirement text. Therefore, the present disclosure rewrites and expands the requirement text to improve the image generation effect.

[0110] Figure 6 It is a flowchart of an image generation method provided according to an embodiment of the present disclosure.

[0111] like Figure 6 As shown, the image generation method includes:

[0112] Step 601, obtaining image generation requirement information.

[0113] Step 602: Determine a corresponding target image generation method according to the image generation requirement information.

[0114] Step 603: determine whether the image generation requirement information requires a reference image query.

[0115] Among them, when performing image generation, not all image generation requirements need to be enhanced through retrieval enhancement. For example, if the image generation requirement information is "draw a little rabbit", this is something that the general Wensheng graph model can handle itself, and there is no need to trigger retrieval enhancement. Based on this, in order to avoid unnecessary complex calculations and resource waste, the present disclosure performs reference image query judgment before performing image generation.

[0116] As an example, the image generation requirement information can be input into the third large model, and the third large model is used to determine whether the image generation requirement information needs to be queried for a reference image. The third large model and the second large model and the first large model can be the same large model or different large models.

[0117] Step 604: When the image generation requirement information requires a reference image query, a first reference image is obtained by querying based on the image generation requirement information.

[0118] Step 605 , based on the image generation requirement information and the first reference image, a target image is generated using a target image generation method.

[0119] It should be noted that the explanation of step 601, step 602, step 604 and step 605 can be found in the relevant description in any embodiment of the present disclosure, and will not be repeated here.

[0120] In the disclosed embodiment, image generation requirement information is obtained; the corresponding target image generation method is determined according to the image generation requirement information; it is determined whether the image generation requirement information needs to be queried for a reference image; in the case where the image generation requirement information needs to be queried for a reference image, a first reference image is queried based on the image generation requirement information; based on the image generation requirement information and the first reference image, a target image generation method is used to generate a target image. The disclosed embodiment determines whether the image generation requirement information needs to be queried for a reference image, and in the case where the image generation requirement information needs to be queried for a reference image, iRAG technology is used for image generation, which not only aligns the timeliness of image generation with the timeliness of image search, avoiding the problem of possible lag of the target image, but also avoids unnecessary complex calculations and waste of resources when retrieval image enhancement is not required, thereby improving the overall image generation efficiency. In addition, since image search covers almost all public knowledge, the disclosed embodiment has no memory capacity bottleneck, and also solves the problem of limited memory capacity of AI image generation models.

[0121] Figure 7 It is a flowchart of an image generating method provided according to another embodiment of the present disclosure.

[0122] like Figure 7 As shown, the image generation method includes:

[0123] Step 701, obtaining image generation requirement information.

[0124] Step 702: Determine a corresponding target image generation method according to the image generation requirement information.

[0125] In addition to the precise image generation method and the highly generalized image generation method, the present disclosure also provides a general text-generated image generation method and an image-generated image generation method for protection. According to the image generation requirement information, a matching target image generation method can be selected from the aforementioned image generation methods.

[0126] For example, assuming that the image generation requirement information is "draw a little rabbit", the corresponding target image generation method can be a general graphic method; assuming that the image generation requirement information is "draw a Flemish giant rabbit", the corresponding target image generation method can be a precise graphic method.

[0127] As an example, obtain the requirement text in the image generation requirement information; input the requirement text into the first large model to perform subject modification intention detection, and obtain the target subject modification intention corresponding to the image generation requirement information, wherein the target subject modification intention is used to indicate whether the subject in the first reference image needs to be modified; based on the mapping relationship between the subject modification intention and the image generation method, obtain the target image generation method corresponding to the target subject modification intention.

[0128] Step 703: Obtain the requirement text in the image generation requirement information.

[0129] Step 704: understand the requirement text and obtain the requirement type corresponding to the image generation requirement information.

[0130] The requirement type may be used to indicate the type of knowledge involved in the image generation requirement information. For example, the requirement type includes simple knowledge, professional knowledge, long-tail knowledge, time-sensitive knowledge, and the like.

[0131] As an example, the requirement text can be input into the fourth model for requirement understanding to obtain the requirement type corresponding to the image generation requirement information. The fourth model and the third model, the second model, and the first model can be the same model or different models.

[0132] Step 705: Based on the requirement type, determine whether the image generation requirement information needs to perform a reference image query.

[0133] There is a corresponding relationship between the demand type and whether a reference image query is required. For example, simple knowledge does not require a reference image query, that is, there is no need to obtain the first reference image. Professional knowledge, long-tail knowledge, and time-sensitive knowledge require a reference image query, that is, the first reference image needs to be obtained.

[0134] When processing image generation requirements, considering that not all requirements require retrieval enhancement to improve the effect, the present disclosure adopts an intelligent strategy, that is, flexibly deciding whether to perform reference image query according to the type of requirement. By accurately judging the requirements, the present disclosure can implement retrieval enhancement in a targeted manner, which not only avoids excessive retrieval enhancement load, but also achieves optimal resource allocation while meeting user needs, avoiding unnecessary resource waste.

[0135] Step 706: When the image generation requirement information requires a reference image query, a first reference image is obtained by querying based on the image generation requirement information.

[0136] As an example, the image subject information included in the image generation requirement information is obtained; based on the image subject information, an image query is performed in a set image library to obtain a first reference image.

[0137] As another example, image subject information included in image generation requirement information is obtained; based on the image subject information, an image query is performed in a set image library to obtain a candidate reference image; image quality of the candidate reference image is obtained, and a set quality requirement corresponding to the image generation requirement information is obtained; and the candidate reference images are screened according to the image quality and the set quality requirement to obtain a first reference image.

[0138] Step 707 : Generate a target image by adopting a target image generation method based on the image generation requirement information and the first reference image.

[0139] As an example, the required text corresponds to a text feature, and in response to the target subject modification intention indicating that the subject in the first reference image does not need to be modified, feature extraction is performed on the first reference image to obtain first image features, and a subject segmentation image in the first reference image is obtained; the first image features and text features are input into a first raw image model to obtain a background image and subject layout information; according to the subject layout information, the subject segmentation image and the background image are fused to obtain a target image.

[0140] As another example, the requirement text corresponds to a text feature. In response to the target subject modification intention indicating that the subject in the first reference image needs to be modified, feature extraction is performed on the first reference image to obtain a second image feature. Based on the description objects corresponding to the text feature and each sub-feature in the second image feature, the text feature and the second image feature are interlaced and spliced ​​to obtain a text-image interlaced feature. The text-image interlaced feature is input into the second raw image model to obtain the target image.

[0141] As a possible implementation method of the embodiment of the present disclosure, based on the description objects corresponding to the text features and each sub-feature in the second image features, the text features and the second image features are interlaced and spliced ​​to obtain text-image interlaced features, including: the image generation requirement information includes a second reference image input by the user, and the second reference image is extracted to obtain a third image feature; based on the description objects corresponding to the text features, the second image features and each sub-feature in the third image features, the text features, the second image features and the third image features are interlaced and spliced ​​to obtain the text-image interlaced features.

[0142] As an example, the requirement text in the image generation requirement information is obtained; the requirement text is rewritten and expanded to obtain the target requirement text; based on the target requirement text and the first reference image, a target image generation method is adopted to generate the target image.

[0143] As an example, context information of image generation requirement information is obtained; when the image generation requirement information includes a second reference image input by a user, description information of the second reference image is obtained; at least one item of the context information and the description information, as well as the requirement text, are input into the second largest model for rewriting and expansion to obtain the target requirement text.

[0144] Step 708: When the image generation requirement information does not require a reference image query, a target image generation method is adopted based on the image generation requirement information to generate a target image.

[0145] Without the need for reference image query, the present disclosure can also efficiently and accurately complete image generation tasks based on image generation requirement information, meet diverse image generation requirements, and thus broaden the application scenarios of the present disclosure.

[0146] As an example, if the detected subject modification intention indicates that there is no need to modify the subject in the second reference image, the target image generation method can be a precise raw image method, specifically, feature extraction is performed on the second reference image to obtain a fourth image feature, and a subject segmentation image in the second reference image is obtained; the fourth image feature and the text feature are input into the first raw image model to obtain the background image and subject layout information; according to the subject layout information, the subject segmentation image and the background image are fused to obtain the target image.

[0147] As another example, if the detected subject modification intention indicates that the subject in the second reference image needs to be modified, the target image generation method can be a highly generalized image generation method. Specifically, feature extraction is performed on the second reference image to obtain the fifth image feature; based on the description objects corresponding to the text features and each sub-feature in the fifth image feature, the text features and the fifth image features are interlaced and spliced ​​to obtain image-text interlaced features; the image-text interlaced features are input into the second image generation model to obtain the target image.

[0148] It should be noted that the explanations of step 701, step 702, step 703, step 706 and step 707 can be found in the relevant descriptions in any embodiment of the present disclosure and will not be repeated here.

[0149] In the embodiment of the present disclosure, the demand text in the image generation demand information is obtained; the demand text is understood to obtain the demand type corresponding to the image generation demand information; based on the demand type, it is determined whether the image generation demand information needs to be queried for a reference image; in the case where the image generation demand information does not need to be queried for a reference image, a target image generation method is adopted based on the image generation demand information to generate a target image; in the case where the image generation demand information needs to be queried for a reference image, a first reference image is obtained by querying based on the image generation demand information; based on the image generation demand information and the first reference image, a target image generation method is adopted to generate a target image. Considering that not all demands need to be enhanced by retrieval enhancement, the present disclosure flexibly determines whether to obtain the first reference image according to the demand type. By accurately judging the demand, the present disclosure can implement retrieval enhancement in a targeted manner, which not only avoids the overload of the retrieval enhancement system, but also achieves the optimal allocation of resources while meeting user needs, avoiding unnecessary waste of resources.

[0150] Figure 8 It is a structural schematic diagram of an image generating device provided according to an embodiment of the present disclosure.

[0151] like Figure 8 As shown, the image generation device includes an acquisition module 801, a determination module 802, a query module 803 and a generation module 804, wherein:

[0152] The acquisition module 801 is used to acquire image generation requirement information; the determination module 802 is used to determine the corresponding target image generation method according to the image generation requirement information; the query module 803 is used to query and obtain the first reference image based on the image generation requirement information; the generation module 804 is used to generate the target image based on the image generation requirement information and the first reference image by adopting the target image generation method.

[0153] As a possible implementation of the embodiment of the present disclosure, the query module 803 is used to: obtain image subject information included in the image generation requirement information; and perform image query in a set image library based on the image subject information to obtain a first reference image.

[0154] As a possible implementation method of the embodiment of the present disclosure, the query module 803 is used to: based on the image subject information, perform image query in the set image library to obtain candidate reference images; obtain the image quality of the candidate reference images, and obtain the set quality requirements corresponding to the image generation requirement information; according to the image quality and the set quality requirements, screen the candidate reference images to obtain the first reference image.

[0155] As a possible implementation method of an embodiment of the present disclosure, the determination module 802 is used to: obtain the requirement text in the image generation requirement information; input the requirement text into the first large model to perform subject modification intention detection, and obtain the target subject modification intention corresponding to the image generation requirement information, wherein the target subject modification intention is used to indicate whether the subject in the first reference image needs to be modified; based on the mapping relationship between the subject modification intention and the image generation method, obtain the target image generation method corresponding to the target subject modification intention.

[0156] As a possible implementation method of the embodiment of the present disclosure, the requirement text corresponds to text features, and the generation module 804 is used to: in response to the target subject modification intention indicating that the subject in the first reference image does not need to be modified, extract features from the first reference image to obtain first image features, and obtain a subject segmentation image in the first reference image; input the first image features and text features into the first raw image model to obtain the background image and subject layout information; and fuse the subject segmentation image and the background image according to the subject layout information to obtain the target image.

[0157] As a possible implementation method of the embodiment of the present disclosure, the requirement text corresponds to a text feature, and the generation module 804 is used to: in response to the target subject modification intention indicating that the subject in the first reference image needs to be modified, extract features from the first reference image to obtain second image features; based on the description objects corresponding to the text features and each sub-feature in the second image features, interlace and splice the text features and the second image features to obtain text-image interlace features; input the text-image interlace features into the second raw image model to obtain the target image.

[0158] As a possible implementation method of the embodiment of the present disclosure, the image generation requirement information includes a second reference image input by a user, and the generation module 804 is used to: extract features from the second reference image to obtain third image features; based on the description objects corresponding to the sub-features in the text features, the second image features and the third image features, interlace and splice the text features, the second image features and the third image features to obtain text-image interlace features.

[0159] As a possible implementation method of the embodiment of the present disclosure, the generation module 804 is used to: obtain the requirement text in the image generation requirement information; rewrite and expand the requirement text to obtain the target requirement text; and generate the target image based on the target requirement text and the first reference image using the target image generation method.

[0160] As a possible implementation method of an embodiment of the present disclosure, the generation module 804 is used to: obtain context information of image generation requirement information; when the image generation requirement information includes a second reference image input by the user, obtain description information of the second reference image; input at least one item of the context information and the description information, as well as the requirement text into the second large model for rewriting and expansion to obtain the target requirement text.

[0161] Fig. 9 It is a structural schematic diagram of an image generating device provided according to an embodiment of the present disclosure.

[0162] like Fig. 9 As shown, the image generation device includes an acquisition module 901, a determination module 902, a judgment module 903, a query module 904 and a generation module 905, wherein:

[0163] An acquisition module 901 is used to acquire image generation requirement information; a determination module 902 is used to determine a corresponding target image generation method according to the image generation requirement information; a judgment module 903 is used to judge whether a reference image query is required for the image generation requirement information; a query module 904 is used to query and obtain a first reference image based on the image generation requirement information when a reference image query is required for the image generation requirement information; a generation module 905 is used to generate a target image by adopting a target image generation method based on the image generation requirement information and the first reference image.

[0164] As a possible implementation method of the embodiment of the present disclosure, the judgment module 903 is used to: obtain the requirement text in the image generation requirement information; understand the requirement text and obtain the requirement type corresponding to the image generation requirement information; based on the requirement type, determine whether the image generation requirement information needs to perform a reference image query.

[0165] As a possible implementation of the embodiment of the present disclosure, the generation module 905 is further used to: generate a target image based on the image generation requirement information and adopt a target image generation method when the image generation requirement information does not require a reference image query.

[0166] As a possible implementation of the embodiment of the present disclosure, the query module 904 is used to: obtain image subject information included in the image generation requirement information; and perform image query in a set image library based on the image subject information to obtain a first reference image.

[0167] As a possible implementation method of the embodiment of the present disclosure, the query module 904 is used to: based on the image subject information, perform image query in the set image library to obtain candidate reference images; obtain the image quality of the candidate reference images, and obtain the set quality requirements corresponding to the image generation requirement information; according to the image quality and the set quality requirements, screen the candidate reference images to obtain the first reference image.

[0168] As a possible implementation method of an embodiment of the present disclosure, the determination module 902 is used to: obtain the requirement text in the image generation requirement information; input the requirement text into the first large model to perform subject modification intention detection, and obtain the target subject modification intention corresponding to the image generation requirement information, wherein the target subject modification intention is used to indicate whether the subject in the first reference image needs to be modified; based on the mapping relationship between the subject modification intention and the image generation method, obtain the target image generation method corresponding to the target subject modification intention.

[0169] As a possible implementation method of the embodiment of the present disclosure, the requirement text corresponds to text features, and the generation module 905 is used to: in response to the target subject modification intention indicating that the subject in the first reference image does not need to be modified, extract features from the first reference image to obtain first image features, and obtain a subject segmentation image in the first reference image; input the first image features and text features into the first raw image model to obtain the background image and subject layout information; and fuse the subject segmentation image and the background image according to the subject layout information to obtain the target image.

[0170] As a possible implementation method of the embodiment of the present disclosure, the requirement text corresponds to a text feature, and the generation module 905 is used to: in response to the target subject modification intention indicating that the subject in the first reference image needs to be modified, extract features from the first reference image to obtain second image features; based on the description objects corresponding to the text features and each sub-feature in the second image features, interlace and splice the text features and the second image features to obtain text-image interlace features; input the text-image interlace features into the second raw image model to obtain the target image.

[0171] As a possible implementation method of the embodiment of the present disclosure, the image generation requirement information includes a second reference image input by a user, and the generation module 905 is used to: extract features from the second reference image to obtain third image features; based on the description objects corresponding to the sub-features of the text features, the second image features and the third image features, interlace and splice the text features, the second image features and the third image features to obtain text-image interlace features.

[0172] As a possible implementation method of the embodiment of the present disclosure, the generation module 905 is used to: obtain the requirement text in the image generation requirement information; rewrite and expand the requirement text to obtain the target requirement text; and generate the target image based on the target requirement text and the first reference image using the target image generation method.

[0173] As a possible implementation method of an embodiment of the present disclosure, the generation module 905 is used to: obtain context information of image generation requirement information; when the image generation requirement information includes a second reference image input by the user, obtain description information of the second reference image; input at least one item of the context information and the description information, as well as the requirement text into the second large model for rewriting and expansion to obtain the target requirement text.

[0174] It should be noted that the above explanation of the image generating method is also applicable to the image generating device of this embodiment and will not be repeated here.

[0175] Fig.10 is a schematic diagram of an intelligent agent provided according to an embodiment of the present disclosure.

[0176] like Fig.10 As shown, the agent 1000 may include: an input module 1001 , a processing module 1002 and an output module 1003 .

[0177] Among them, as an example, the input module 1001 is used to obtain image generation requirement information; the processing module 1002 is used to determine the corresponding target image generation method according to the image generation requirement information; based on the image generation requirement information, query to obtain the first reference image; based on the image generation requirement information and the first reference image, adopt the target image generation method to generate the target image; the output module 1003 is used to output the target image.

[0178] As another example, the input module 1001 is used to obtain image generation requirement information; the processing module 1002 is used to determine the corresponding target image generation method according to the image generation requirement information; determine whether the image generation requirement information requires a reference image query; if the image generation requirement information requires a reference image query, query and obtain a first reference image based on the image generation requirement information; based on the image generation requirement information and the first reference image, generate a target image using a target image generation method; and the output module 1003 is used to output the target image.

[0179] According to an embodiment of the present disclosure, the present disclosure also provides an intelligent agent system, a readable storage medium and a computer program product.

[0180] Fig.11 A schematic block diagram of an example intelligent agent system 1100 that can be used to implement an embodiment of the present disclosure is shown. The intelligent agent system 1100 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The intelligent agent system can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0181] like Fig.11 As shown, the agent system 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 1102 or a computer program loaded from a storage unit 1108 to a RAM (Random Access Memory) 1103. In the RAM 1103, various programs and data required for the operation of the agent system 1100 can also be stored. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An I / O (Input / Output) interface 1105 is also connected to the bus 1104.

[0182] Multiple components in the agent system 1100 are connected to the I / O interface 1105, including: an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the agent system 1100 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0183] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 1101 performs the various methods and processes described above, such as the image generation method. For example, in some embodiments, the image generation method may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the agent system 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the image generation method described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the image generation method in any other appropriate manner (eg, by means of firmware).

[0184] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0185] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0186] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0187] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0188] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0189] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.

[0190] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.

[0191] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0192] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating an image, comprising: Obtaining image generation requirement information; Determine a corresponding target image generation method according to the image generation requirement information; Based on the image generation requirement information, query and obtain a first reference image; Based on the image generation requirement information and the first reference image, the target image is generated using the target image generation method.

2. The method according to claim 1, wherein: The querying and obtaining a first reference image based on the image generation requirement information includes: Acquire the image subject information included in the image generation requirement information; Based on the image subject information, an image query is performed in a set image library to obtain the first reference image.

3. The method according to claim 2, wherein: The step of performing an image search in a set image library based on the image subject information to obtain the first reference image includes: Based on the image subject information, an image query is performed in a set image library to obtain a candidate reference image; Obtaining the image quality of the candidate reference image, and obtaining the set quality requirement corresponding to the image generation requirement information; The candidate reference images are screened according to the image quality and the set quality requirement to obtain the first reference image.

4. The method according to claim 1, wherein: The determining a corresponding target image generation method according to the image generation requirement information includes: Obtaining the requirement text in the image generation requirement information; Input the requirement text into the first large model to perform subject modification intention detection, and obtain the target subject modification intention corresponding to the image generation requirement information, wherein the target subject modification intention is used to indicate whether the subject in the first reference image needs to be modified; Based on the mapping relationship between the subject modification intention and the image generation method, a target image generation method corresponding to the target subject modification intention is obtained.

5. The method according to claim 4, wherein: The requirement text corresponds to a text feature, and the target image is generated based on the image generation requirement information and the first reference image, using the target image generation method, including: In response to the target subject modification intention indicating that the subject in the first reference image does not need to be modified, extracting features from the first reference image to obtain first image features, and acquiring a subject segmentation image in the first reference image; Inputting the first image feature and the text feature into a first raw image model to obtain a background image and main body layout information; The subject segmentation image and the background image are fused according to the subject layout information to obtain the target image.

6. The method according to claim 4, wherein: The requirement text corresponds to a text feature, and the target image is generated based on the image generation requirement information and the first reference image, using the target image generation method, including: In response to the target subject modification intention indicating that the subject in the first reference image needs to be modified, extracting features from the first reference image to obtain second image features; Based on the description objects corresponding to the sub-features in the text feature and the second image feature, the text feature and the second image feature are interlaced and spliced ​​to obtain an image-text interlaced feature; The image-text interlaced feature is input into a second image generation model to obtain the target image.

7. The method according to claim 6, wherein: The image generation requirement information includes a second reference image input by a user, and the description object corresponding to each sub-feature in the text feature and the second image feature is based on the text feature and the second image feature, and the text feature and the second image feature are interlaced and spliced ​​to obtain the image-text interlaced feature, including: performing feature extraction on the second reference image to obtain third image features; Based on the description objects corresponding to the sub-features in the text feature, the second image feature and the third image feature, the text feature, the second image feature and the third image feature are interlaced and spliced ​​to obtain the text-image interlaced feature.

8. The method according to claim 1, wherein: The step of generating a target image based on the image generation requirement information and the first reference image by adopting the target image generation method includes: Obtaining the requirement text in the image generation requirement information; Rewrite and expand the demand text to obtain a target demand text; Based on the target requirement text and the first reference image, the target image is generated using the target image generation method.

9. The method according to claim 8, wherein: The step of rewriting and expanding the requirement text to obtain a target requirement text includes: Acquire context information of the image generation requirement information; In a case where the image generation requirement information includes a second reference image input by a user, acquiring description information of the second reference image; At least one of the context information and the description information, and the requirement text are input into the second largest model for rewriting and expansion to obtain the target requirement text.

10. A method for generating an image, comprising: Obtaining image generation requirement information; Determine a corresponding target image generation method according to the image generation requirement information; Determining whether the image generation requirement information requires a reference image query; In a case where the image generation requirement information requires a reference image query, querying and obtaining a first reference image based on the image generation requirement information; Based on the image generation requirement information and the first reference image, the target image is generated using the target image generation method.

11. The method according to claim 10, wherein: The determining whether the image generation requirement information needs to be queried for a reference image includes: Obtaining the requirement text in the image generation requirement information; Understanding the requirement text to obtain the requirement type corresponding to the image generation requirement information; Based on the requirement type, it is determined whether the image generation requirement information requires a reference image query.

12. The method according to claim 10, wherein: The method further comprises: In the case that the image generation requirement information does not require a reference image query, the target image generation method is adopted based on the image generation requirement information to generate the target image.

13. The method according to claim 10, wherein: The querying and obtaining a first reference image based on the image generation requirement information includes: Acquire the image subject information included in the image generation requirement information; Based on the image subject information, an image query is performed in a set image library to obtain the first reference image.

14. The method according to claim 13, wherein: The step of performing an image search in a set image library based on the image subject information to obtain the first reference image includes: Based on the image subject information, an image query is performed in a set image library to obtain a candidate reference image; Obtaining the image quality of the candidate reference image, and obtaining the set quality requirement corresponding to the image generation requirement information; The candidate reference images are screened according to the image quality and the set quality requirement to obtain the first reference image.

15. The method according to claim 10, wherein: The determining a corresponding target image generation method according to the image generation requirement information includes: Obtaining the requirement text in the image generation requirement information; Input the requirement text into the first large model to perform subject modification intention detection, and obtain the target subject modification intention corresponding to the image generation requirement information, wherein the target subject modification intention is used to indicate whether the subject in the first reference image needs to be modified; Based on the mapping relationship between the subject modification intention and the image generation method, a target image generation method corresponding to the target subject modification intention is obtained.

16. An image generating device, comprising: An acquisition module, used to acquire image generation requirement information; A determination module, used to determine a corresponding target image generation method according to the image generation requirement information; A query module, used for generating demand information based on the image, and querying to obtain a first reference image; A generating module is used to generate a target image based on the image generation requirement information and the first reference image and using the target image generation method.

17. The device according to claim 16, wherein: The query module is used to: Acquire the image subject information included in the image generation requirement information; Based on the image subject information, an image query is performed in a set image library to obtain the first reference image.

18. The device according to claim 17, wherein: The query module is used to: Based on the image subject information, an image query is performed in a set image library to obtain a candidate reference image; Obtaining the image quality of the candidate reference image, and obtaining the set quality requirement corresponding to the image generation requirement information; The candidate reference images are screened according to the image quality and the set quality requirement to obtain the first reference image.

19. The device according to claim 16, wherein: The determination module is used for: Obtaining the requirement text in the image generation requirement information; Input the requirement text into the first large model to perform subject modification intention detection, and obtain the target subject modification intention corresponding to the image generation requirement information, wherein the target subject modification intention is used to indicate whether the subject in the first reference image needs to be modified; Based on the mapping relationship between the subject modification intention and the image generation method, a target image generation method corresponding to the target subject modification intention is obtained.

20. The device according to claim 19, wherein The requirement text corresponds to a text feature, and the generating module is used for: In response to the target subject modification intention indicating that the subject in the first reference image does not need to be modified, extracting features from the first reference image to obtain first image features, and acquiring a subject segmentation image in the first reference image; Inputting the first image feature and the text feature into a first raw image model to obtain a background image and main body layout information; The subject segmentation image and the background image are fused according to the subject layout information to obtain the target image.

21. The device according to claim 19, wherein The requirement text corresponds to a text feature, and the generating module is used for: In response to the target subject modification intention indicating that the subject in the first reference image needs to be modified, extracting features from the first reference image to obtain second image features; Based on the description objects corresponding to the sub-features in the text feature and the second image feature, the text feature and the second image feature are interlaced and spliced ​​to obtain an image-text interlaced feature; The image-text interlaced feature is input into a second image generation model to obtain the target image.

22. The device according to claim 21, wherein The image generation requirement information includes a second reference image input by a user, and the generation module is used to: performing feature extraction on the second reference image to obtain third image features; Based on the description objects corresponding to the sub-features in the text feature, the second image feature and the third image feature, the text feature, the second image feature and the third image feature are interlaced and spliced ​​to obtain the text-image interlaced feature.

23. The device according to claim 16, wherein: The generation module is used for: Obtaining the requirement text in the image generation requirement information; Rewrite and expand the demand text to obtain a target demand text; Based on the target requirement text and the first reference image, the target image is generated using the target image generation method.

24. The device according to claim 23, wherein: The generation module is used for: Acquire context information of the image generation requirement information; In a case where the image generation requirement information includes a second reference image input by a user, acquiring description information of the second reference image; At least one of the context information and the description information, and the requirement text are input into the second largest model for rewriting and expansion to obtain the target requirement text.

25. An image generating device, comprising: An acquisition module, used to acquire image generation requirement information; A determination module, used to determine a corresponding target image generation method according to the image generation requirement information; A judgment module, used to judge whether the image generation requirement information needs to be queried for a reference image; A query module, configured to query and obtain a first reference image based on the image generation requirement information when the image generation requirement information requires a reference image query; A generating module is used to generate a target image based on the image generation requirement information and the first reference image and using the target image generation method.

26. The device according to claim 25, wherein The judging module is used for: Obtaining the requirement text in the image generation requirement information; Understanding the requirement text to obtain the requirement type corresponding to the image generation requirement information; Based on the requirement type, it is determined whether the image generation requirement information requires a reference image query.

27. The device according to claim 25, wherein: The generation module is also used for: In the case that the image generation requirement information does not require a reference image query, the target image generation method is adopted based on the image generation requirement information to generate the target image.

28. The device according to claim 25, wherein The query module is used to: Acquire the image subject information included in the image generation requirement information; Based on the image subject information, an image query is performed in a set image library to obtain the first reference image.

29. The device according to claim 28, wherein The query module is used to: Based on the image subject information, an image query is performed in a set image library to obtain a candidate reference image; Obtaining the image quality of the candidate reference image, and obtaining the set quality requirement corresponding to the image generation requirement information; The candidate reference images are screened according to the image quality and the set quality requirement to obtain the first reference image.

30. The device according to claim 25, wherein: The determination module is used for: Obtaining the requirement text in the image generation requirement information; Input the requirement text into the first large model to perform subject modification intention detection, and obtain the target subject modification intention corresponding to the image generation requirement information, wherein the target subject modification intention is used to indicate whether the subject in the first reference image needs to be modified; Based on the mapping relationship between the subject modification intention and the image generation method, a target image generation method corresponding to the target subject modification intention is obtained.

31. An intelligent agent, comprising: An input module is used to obtain image generation requirement information; A processing module, used to determine a corresponding target image generation method according to the image generation requirement information; Based on the image generation requirement information, query and obtain a first reference image; Based on the image generation requirement information and the first reference image, using the target image generation method to generate a target image; An output module is used to output the target image.

32. An intelligent agent, comprising: An input module is used to obtain image generation requirement information; A processing module, used to determine a corresponding target image generation method according to the image generation requirement information; Determine whether the image generation requirement information requires a reference image query; if the image generation requirement information requires a reference image query, query and obtain a first reference image based on the image generation requirement information; Based on the image generation requirement information and the first reference image, using the target image generation method to generate a target image; An output module is used to output the target image.

33. An intelligent agent system comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-9 or 10-15.

34. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method of any one of claims 1-9 or 10-15.

35. A computer program product, comprising a computer program, which, when executed by a processor, implements the method of any one of claims 1-9 or 10-15.

Images