Graph generation method, device and equipment based on text graph inference model and storage medium

Abstract

The invention provides a graph generation method and device based on a text graph reasoning model, equipment and a storage medium, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like, and can be applied to scenes such as content generation based on artificial intelligence. The specific implementation scheme is as follows: acquiring a map generation request of a user; reasoning the map generation request by adopting a pre-trained text map reasoning model to obtain a retrieval instruction adopted when a retrieval enhancement generation tool is triggered; planning a graph generation step based on the retrieval instruction by adopting the text graph reasoning model; and generating a target picture based on the picture generation step.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computers, in particular to the technical field of artificial intelligence, and more particularly to the technical fields of computer vision, deep learning, large models, and the like, which can be applied to scenarios such as content generation based on artificial intelligence. In particular, a picture generation method and device based on a text-to-picture reasoning model, equipment, and a storage medium are disclosed. BACKGROUND

[0002] In recent years, with the vigorous development of text-to-picture technology products, the technical application of text-to-picture models has attracted widespread attention from ordinary users. At the same time, with the development and breakthrough of reasoning technology in the model, reasoning models are also attracting widespread attention.

[0003] Existing text-to-picture technology focuses on the optimization and iteration of text-to-picture models such as diffusion models, augmented reality (AR) models, and AR+diffusion models. SUMMARY

[0004] The present disclosure provides a picture generation method and device based on a text-to-picture reasoning model, equipment, and a storage medium.

[0005] According to an aspect of the present disclosure, a picture generation method based on a text-to-picture reasoning model is provided, comprising:

[0006] obtaining a picture generation request of a user;

[0007] using a pre-trained text-to-picture reasoning model to reason the picture generation request and obtain a retrieval instruction used when triggering a retrieval enhancement generation tool;

[0008] using the text-to-picture reasoning model to plan picture generation steps based on the retrieval instruction;

[0009] generating a target picture based on the picture generation steps.

[0010] According to another aspect of the present disclosure, a picture generation device based on a text-to-picture reasoning model is provided, comprising:

[0011] an obtaining module configured to obtain a picture generation request of a user;

[0012] a reasoning module configured to use a pre-trained text-to-picture reasoning model to reason the picture generation request and obtain a retrieval instruction used when triggering a retrieval enhancement generation tool;

[0013] a step planning module configured to use the text-to-picture reasoning model to plan picture generation steps based on the retrieval instruction;

[0014] The picture generation module is configured to generate a target picture based on the picture generation step.

[0015] According to still another aspect of the present disclosure, an electronic device is provided, comprising:

[0016] at least one processor; and

[0017] a memory connected with the at least one processor; wherein

[0018] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of the aspects and any possible implementation manners described above.

[0019] According to still another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, the computer instructions being used to cause the computer to perform the method of the aspects and any possible implementation manners described above.

[0020] According to still another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method of the aspects and any possible implementation manners described above.

[0021] According to still another aspect of the present disclosure, an intelligent agent is provided, comprising:

[0022] The input module is configured to receive a picture generation request of a user;

[0023] The processing module is configured to determine a target task based on the input information received by the input module, determine a picture generation reasoning model based on the target task, execute the method of the aspects and any possible implementation manners described above by calling the picture generation reasoning model, and obtain output information;

[0024] The output module is configured to output the output information obtained by the processing module.

[0025] According to the technology of the present disclosure, a brand new picture generation manner can be provided, which can not only generate pictures, but also provide detailed reasoning information and picture generation steps, and can effectively enrich the picture generation manner.

[0026] It should be understood that the contents described in this part are not intended to identify the key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0028] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0029] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0030] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0031] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0032] Figure 5 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0033] Figure 6 is a schematic diagram according to a sixth embodiment of the present disclosure;

[0034] Figure 7 is a block diagram of an electronic device for implementing the method of the embodiments of the present disclosure. DETAILED DESCRIPTION

[0035] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to help in understanding, and should be considered as merely exemplary. Thus, those skilled in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0036] Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present disclosure.

[0037] It should be noted that the terminal device involved in the embodiments of the present disclosure can include, but is not limited to, a mobile phone, a personal digital assistant (PDA), a wireless handheld device, a tablet computer, and the like. The display device can include, but is not limited to, a personal computer, a television, and the like.

[0038] In addition, the term "and / or" in this paper is only a description of the association relationship between the associated objects, which means that there can be three relationships, for example, A and / or B, which can represent the existence of A alone, the existence of A and B together, and the existence of B alone. In addition, the character " / " in this paper generally represents an "or" relationship between the associated objects before and after it.

[0039] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure; as Figure 1 shown, the embodiment provides a picture generation method of a text-to-picture reasoning model, which can specifically include the following steps:

[0040] S101, obtaining a picture generation request of a user;

[0041] S102, using a pre-trained text-to-picture reasoning model to reason the picture generation request, and obtaining a retrieval instruction used when triggering a retrieval-augmented generation (RAG) tool; S103, using the text-to-picture reasoning model to plan a picture generation step based on the retrieval instruction;

[0042] S104, generating a target picture based on the picture generation step.

[0043] The execution subject of the picture generation method of the text-to-picture reasoning model of the embodiment is a picture generation device of the text-to-picture reasoning model, which can be an electronic entity or an application product integrated with software.

[0044] The text-to-picture reasoning model of the embodiment is different from the existing text-to-picture model in that the existing text-to-picture model directly generates a picture based on text, and the reasoning process of how the model generates the picture at the bottom layer is unknown. However, the text-to-picture reasoning model of the embodiment can reason after obtaining the picture generation request, obtain the retrieval instruction used when triggering the RAG tool as reasoning information, and further plan the picture generation step based on the retrieval instruction, which can specifically reflect the picture generation process before generating the picture.

[0045] In the embodiment, the text-to-picture reasoning model can be a multi-modal visual language model (VLM), which can receive the picture generation request of the user, reason and analyze, obtain the retrieval instruction used when triggering the RAG tool, plan the picture generation step, and finally generate the picture, which can specifically reason and analyze the picture generation process, so that the user can personally feel how the generated picture is reasoned and analyzed and generated.

[0046] The picture generation method of the text-to-picture reasoning model of the embodiment can infer the picture generation request of a user by using the text-to-picture reasoning model, acquire a search instruction used when the RAG tool is triggered, further plan a picture generation step based on the search instruction, and then generate a target picture based on the picture generation step. The technical solution of the embodiment can provide a brand-new picture generation method, which can not only generate a picture, but also provide detailed reasoning information and a picture generation step, and can effectively enrich the picture generation method. Meanwhile, the user can be more clearly aware of how the picture is inferred and analyzed, and thus the use experience of the user can be effectively enhanced. Moreover, the technical solution of the embodiment can effectively improve the accuracy of the generated target picture by using detailed reasoning and planning a picture generation step.

[0047] Figure 2 is a schematic diagram according to the second embodiment of the present disclosure; the picture generation method of the text-to-picture reasoning model of the embodiment, in the above Figure 1 The technical solution of the embodiment is further described in more detail. As shown in Figure 2 The picture generation method of the text-to-picture reasoning model of the embodiment can specifically include the following steps:

[0048] S201, acquiring a picture generation request of a user;

[0049] In the embodiment, the picture generation request of the user can only include text request information, for example, the text request information can be: please help generate a night scene picture with A building as the background, please help generate a picture of a cartoon-style character B, and the like.

[0050] Alternatively, in another scenario, the picture generation request of the embodiment can not only include text request information, but also include a basic picture. For example, in this scenario, the text request information corresponding to the user can be: please help increase the resolution of the basic picture, please help add an object C to the basic picture, please help modify the color of an animal D in the basic picture to color E, and the like.

[0051] In addition, the picture generation request of the embodiment can also be a complex request for generating a series of pictures, for example, "generate a set of consistent story illustrations related to event Q, which need to have strong coherence".

[0052] S202, inferring the picture generation request by using a text-to-picture reasoning model, and acquiring a picture generation requirement;

[0053] Specifically, the text request information in the picture generation request is mainly inferred to acquire the picture generation requirement, that is, the corresponding intention.

[0054] The raw image requirement in this embodiment expresses its intent more clearly than the raw image request. If the raw image request only includes text request information, the corresponding raw image requirement also includes only text information. If the raw image request includes text request information and reference images, the corresponding raw image requirement also includes text information and images.

[0055] S203: Analyze the raw image requirements using a text-image reasoning model to determine whether to trigger the RAG tool and the type of reference image to be recalled by the RAG tool;

[0056] The RAG tool of this embodiment can be considered as a third-party tool independent of the text-graph reasoning model. The RAG has a huge image source to support its retrieval.

[0057] The type of reference image requested for recall can be obtained based on the analysis of the raw image requirements. Specifically, the raw image requirements can be analyzed first. If it is determined that the RAG tool needs to be triggered to retrieve the reference image, the raw image requirements can be analyzed again to determine the type of reference image to be obtained. For example, in this embodiment, the type can include a subject type or a style type. For example, if the raw image requirement is: generate a portrait with the gate of M University as the background; at this time, the raw image reasoning model can determine through reasoning analysis that the RAG tool needs to be triggered to obtain a reference image of the gate of M University, and it can be determined that the image of the gate of M University to be recalled is a subject type image. For another example, if the raw image requirement is: refer to the cartoon image of the character W in the cartoon Q to generate a cartoon image for the base image; at this time, the raw image reasoning model can determine through reasoning analysis that the RAG tool needs to be triggered to obtain a cartoon image of the character W in the cartoon Q, and it can be determined that the cartoon image of the character W to be recalled is a style type image.

[0058] S204, detecting whether the RAG tool needs to be triggered; if not, executing step S205; if yes, executing step S206;

[0059] S205: Using the text-image reasoning model, based on the basic images and the image generation requirements, plan the image generation steps and display them; then execute step S210;

[0060] S206. Using the text-image reasoning model, based on the image requirements and the type of reference images, generate and display a search instruction; and execute step S207.

[0061] If the reference picture to be recalled is determined to be a subject picture based on the picture generation requirement, the corresponding generated search instruction needs to reflect the subject picture search enhancement; if the reference picture to be recalled is determined to be a style picture based on the picture generation requirement, the corresponding generated search instruction needs to reflect the style picture search enhancement. In short, the type of the reference picture to be recalled needs to be reflected on the constructed search instruction, so as to be able to recall more accurate reference pictures.

[0062] S207, calling the RAG tool based on the search instruction to obtain the reference picture; performing step S208;

[0063] In this embodiment, through the above steps, the picture generation request can be analyzed in detail, and when the RAG tool needs to be called to obtain the reference picture, the corresponding search instruction is generated, and then the reference picture is obtained, which provides an effective basis for generating accurate target pictures.

[0064] S208, using the picture generation reasoning model to detect whether the reference picture meets the requirements based on the reference picture and the search instruction; if it meets the requirements, performing step S209; if it does not meet the requirements, returning to step S206 to regenerate the search instruction;

[0065] Alternatively, it can also be directly returned to step S207 to call the RAG tool again based on the previous search instruction to obtain the reference picture.

[0066] For example, this step S208 can include the following steps when implemented:

[0067] (a) using the picture generation reasoning model to detect whether the relevance of the reference picture and the search instruction meets the requirements;

[0068] Specifically, the picture generation reasoning model can be used to obtain the features of the reference picture; the picture generation reasoning model is used to obtain the features of the search instruction; then the feature similarity of the two is obtained based on the features of the reference picture and the features of the search instruction; if it is determined whether the feature similarity is greater than a preset similarity threshold, if it is greater, it is determined that the relevance of the reference picture and the search instruction meets the requirements; otherwise, it is considered that the relevance of the reference picture and the search instruction does not meet the requirements.

[0069] (b) using the picture generation reasoning model to detect whether the quality of the reference picture meets the requirements;

[0070] In this embodiment, the quality detection of the reference picture can be detected from the clarity, integrity and aesthetic degree of the picture.

[0071] For example, the text-to-image reasoning model can be used to obtain the definition, completeness and aesthetic degree of the reference picture respectively, and then the definition, completeness and aesthetic degree of the reference picture are weighted and summed according to the preset weight ratio to obtain a quality score. Then it is judged whether the quality score is greater than a preset score; if it is greater, it is determined that the reference picture quality is qualified; otherwise, it is determined that the reference picture quality is not reasonable.

[0072] Or the definition threshold, the completeness threshold and the aesthetic threshold can also be set respectively; it is detected whether the definition of the reference picture is greater than the definition threshold, whether the aesthetic degree of the reference picture is greater than the aesthetic threshold, and whether the completeness of the reference picture is greater than the completeness threshold; only when the definition of the reference picture is greater than the definition threshold, the aesthetic degree is greater than the aesthetic threshold, and the completeness is greater than the completeness threshold, it is considered that the quality of the reference picture meets the requirements.

[0073] (c) When the relevance of the reference picture and the search instruction meets the requirements, and the quality of the reference picture also meets the requirements, it is determined that the reference picture meets the requirements.

[0074] In this embodiment, whether the reference picture meets the requirements is detected from two aspects of the relevance of the reference picture and the search instruction, and the quality of the reference picture. As long as one of them does not meet the requirements, it is considered that the reference picture does not meet the requirements and needs to be recalled.

[0075] In actual application, one reference picture can be requested in one text-to-image request, or more than two reference pictures can be requested, and the implementation principle is the same, which will not be repeated here.

[0076] In this embodiment, by using the above technology, the reference picture obtained from the RAG can be subjected to very strict quality detection, ensuring the accuracy of the obtained reference picture, and thus the accuracy of the generated target picture can be effectively improved.

[0077] S209, using a text-to-image reasoning model, planning the text-to-image steps based on the reference picture and the text-to-image demand, and displaying; performing step S210;

[0078] For example, when this step is implemented, it can include the following steps:

[0079] (1) Using a text-to-image reasoning model, based on the reference picture and the text-to-image demand, determining the identification of the called target text-to-image tool and the parameter information required for calling the target text-to-image tool;

[0080] In this embodiment, the text-to-image tool is also a third-party tool independent of the text-to-image reasoning model, which can be called for text-to-image.

[0081] In this embodiment, the available image generation tools can include: a general text image generation tool, a general image image generation tool, an instruction editing image generation tool, an inpainting image generation tool, an outpainting image generation tool, or a super-resolution model image generation tool.

[0082] The general text image generation tool is used for image generation based on an arbitrary text query input by a user.

[0083] The general image image generation tool is used for image generation based on a reference image uploaded by a user and a text query input by the user.

[0084] The instruction editing image generation tool is used for image generation based on a reference image uploaded by a user and specific editing instructions, and the instruction editing image generation tool needs to ensure very high consistency compared with the general image image generation tool.

[0085] The inpainting image generation tool is used for generating appropriate content based on a specified area in a reference image uploaded by a user. For example, the inpainting image generation tool can include deleting an element in an image and automatically filling the area, replacing the content of a certain area, restoring a damaged image, modifying details such as a face, clothes, and a background of a person, and the like.

[0086] The outpainting image generation tool is used for extending the edge area of an image outward and generating new content based on a reference image uploaded by a user and an image extension instruction query.

[0087] The super-resolution model image generation tool is used for improving the resolution based on a reference image uploaded by a user.

[0088] The above image generation tools in this embodiment are developed and set in advance, and can be called at any time when needed in the use process. The above image generation tools are only examples of the present disclosure, and the technical solution of this embodiment can support the calling of more types of image generation tools in the actual application process, and will not be described one by one here.

[0089] The technical solution of this embodiment can support a variety of image generation tools, can realize text image generation of various needs, and has a very wide application range.

[0090] (2) An image generation step is planned based on a reference image, image generation requirements, an identifier of a target image generation tool, and parameter information required for calling the target image generation tool by using a text image generation reasoning model.

[0091] Specifically, because different inpainting tools have different inpainting manners, in the embodiment, when planning the inpainting step, not only the reference picture and the inpainting requirement are referred to, but also the target generation tool and the parameters required for calling the target generation tool are referred to. That is, in the planning generation step, the reference picture, the inpainting requirement and the parameters required for calling the target inpainting tool are all taken into account, so as to effectively improve the accuracy of the generation step.

[0092] For example, the user's inpainting request is to generate a picture at the entrance of A University, and there is a female student with red clothes and black hair on the right side of the picture. According to the technical solution of the above embodiment, the picture at the entrance of A University, that is, the reference picture, can be obtained, it is determined that the Inpainting inpainting tool can be used, and the determined parameters include red clothes, black hair, female, etc. Combined with all the above information, the planned inpainting step can include: first, adding a female student on the right side of the reference picture; second, setting the hair of the female student to black; third, setting the clothes of the female student to red. It should be noted that the above example of the planned generation step is only an example and does not limit the technical solution of the present disclosure. In specific applications, the inpainting step can be planned according to the reference picture, the inpainting requirement, the identification of the target inpainting tool and the parameter information required for calling the target inpainting tool.

[0093] For example, in order to improve the fidelity of the generated target picture, the picture can be first upgraded to an intermediate resolution lower than the target resolution, and then upgraded one or more times.

[0094] In addition, optionally, for generating a group of pictures of the same theme, the inpainting step can also limit the order of generating a group of pictures, and for each picture, the corresponding inpainting step can also be limited.

[0095] In actual application scenarios, there are many types of inpainting requirements, and the corresponding planned inpainting steps are also very flexible. When used, the text inpainting reasoning model can accurately and reasonably plan the corresponding inpainting step based on the specific situation of each inpainting requirement, and the specific form is not limited in the embodiment.

[0096] Correspondingly, the way of planning the inpainting step in step S205 can also adopt the above way, which will not be described here.

[0097] In the embodiment, by using the text inpainting reasoning model, based on the reference picture, the inpainting requirement, the identification of the target inpainting tool and the parameter information required for calling the target inpainting tool, a very reasonable and accurate inpainting step can be planned, which provides a basis for generating an accurate target picture.

[0098] S210, generate prompt word information based on the graph generation step; execute step S211;

[0099] Since the text-to-graph reasoning model has fused all information such as the reference picture, the graph generation requirement, the identifier of the target graph generation tool, and the parameter information required for calling the target graph generation tool when planning the graph generation step, an optimized prompt word information can be directly generated based on the graph generation step.

[0100] S211, call the target graph generation tool based on the prompt word information to generate a target picture; and display.

[0101] The calling process can be considered as instructing the target graph generation tool to generate a target picture according to the graph generation step and returning.

[0102] The text-to-graph reasoning model and the graph generation method thereof in the embodiment can provide a rich, comprehensive, and complete graph generation reasoning and analysis process, so that the user can understand the entire graph generation process. Moreover, since the embodiment adopts detailed graph generation reasoning and planning of the graph generation step, the accuracy and high fidelity of the generated target picture can be effectively improved. In the embodiment, the intermediate results such as the reasoning information and the generation step can be effectively displayed, so that the user can understand the graph generation process in real time and improve the user experience.

[0103] Figure 3 is a schematic diagram according to the third embodiment of the present disclosure; the text-to-graph reasoning model and the graph generation method thereof in the embodiment are based on the technical solutions of the embodiments described above Figure 2 to further describe the technical solutions of the present disclosure in more detail. As Figure 3 indicated, the text-to-graph reasoning model and the graph generation method thereof in the embodiment can specifically include the following steps:

[0104] S301, generate an evaluation question based on the target picture and the graph generation requirement by using the text-to-graph reasoning model, and display; execute step S302;

[0105] S302, determine the identifier of the target evaluation tool based on the evaluation question by using the text-to-graph reasoning model; execute step S303;

[0106] The evaluation tool in the embodiment can include at least one of the following tools:

[0107] The VLM semantic correlation detection tool is mainly used to evaluate the semantic correlation between the query of the user and the generated target picture by using the VLM, mainly including the judgment of the dimensions such as the subject element, the picture style, the position relationship, the quantity, and the attribute;

[0108] The picture aesthetic detection tool is mainly used for judging the aesthetic degree of the generated target picture through the VLM;

[0109] The text garbled code detection tool is used for judging whether there is garbled code text content in the generated target picture;

[0110] The character limb distortion detection tool is used for judging whether there is a distortion condition of limb parts such as face, hand, leg, etc. in the generated target picture;

[0111] The consistency detection tool is used for detecting the similarity and consistency between the generated target picture and the base picture for the scene containing the base picture;

[0112] The prompt rewriting quality detection is mainly used for judging whether there is information loss in the prompt inference rewriting content in the picture generation process. For example, the prompt here includes the prompt word information generated in the picture generation step.

[0113] Correspondingly, in the embodiment, the generated evaluation question can include one, two or more. The evaluation question can be automatically generated according to the available evaluation tools.

[0114] S303, based on the target picture and the picture generation requirement, the parameter information of calling the target evaluation tool is obtained by using the text-to-picture inference model; step S304 is executed;

[0115] Different evaluation tools need different parameters when performing evaluation work. According to the functions of each evaluation tool, the parameter information required by the evaluation tool to perform its evaluation work is obtained. For example, the parameter information required by the VLM semantic correlation detection tool can include the user's query and the generated target picture. The parameter information required by the picture aesthetic detection tool can only include the generated target picture, and so on, which will not be repeated here.

[0116] S304, based on the parameter information of calling the target evaluation tool and the identification of the target evaluation tool, the target evaluation tool is called to perform evaluation; step S305 is executed;

[0117] S305, the evaluation result of the target evaluation tool is obtained; and displayed; step S306 is executed;

[0118] The specific evaluation result can include specific information obtained by evaluation, such as the result detected by the VLM semantic correlation detection tool, that is, the semantic correlation value between the user's query and the generated target picture, whether it meets the requirement; if the picture beauty detection tool is also included, the corresponding evaluation result also includes whether the picture beauty detection value meets the requirement; correspondingly, if other evaluation tools are also included, the evaluation results of the other evaluation tools also need to be obtained.

[0119] S306, using the text-to-image reasoning model, generating an image evaluation result based on the evaluation result of the target evaluation tool; and displaying; executing step S307;

[0120] Specifically, based on the evaluation result of the target evaluation tool, the final image evaluation result is generated. If multiple target evaluation tools are used for evaluation, the image evaluation result is equivalent to the aggregation of the evaluation results of the multiple target evaluation tools.

[0121] S307, using the text-to-image reasoning model, based on the image evaluation result, detecting whether it is necessary to regenerate the image; if necessary, executing step S308; if not necessary, executing step S309;

[0122] For example, if the image evaluation result includes at least one of the following problems: the semantic correlation between the user's query and the generated target picture is poor, there are garbled characters in the generated target picture, and the similarity and consistency between the generated target picture and the base picture are poor, etc., these problems are more serious, at this time it can be considered that it is necessary to regenerate the image to overcome the problem. If the image evaluation result includes the following problems: the generated target picture is not beautiful enough, or the generated target picture has a distorted body, or the resolution is not enough when the super-resolution model is adjusted, etc., at this time it can be considered that directly modifying the generated picture can overcome the problem, at this time it is not necessary to regenerate the image, but to modify it directly. When modifying, the corresponding image generation tool can be called for modification.

[0123] S308, using the text-to-image reasoning model, optimizing the image generation step based on the reference picture and the image generation requirement, and displaying; returning to the step S210 of the embodiment shown in the above Figure 2 The step S210 of the embodiment shown in the above continues to execute the image generation.

[0124] S309, using the text-to-image reasoning model, based on the image evaluation result, detecting whether it is necessary to modify the target picture; if necessary, executing step S310; if not necessary, determining that the generated target picture meets the requirement, and ending.

[0125] S310, using the text-to-image reasoning model, based on the image evaluation result and the generated target picture, determining the identifier of the called image modification tool; executing step S311;

[0126] S311, using the text-to-image reasoning model, obtaining parameter information of calling the modified image generation tool based on the reference image, the image generation requirement, the target image, and the image generation evaluation result; performing step S312,

[0127] S312, using the text-to-image reasoning model, generating a modification instruction of calling the modified image generation tool based on the reference image, the image generation requirement, the target image, the image generation evaluation result, and the parameter information of calling the modified image generation tool; performing step S313;

[0128] The modified image generation tool is also any one of the optional image generation tools in the above-mentioned embodiments.

[0129] S313, calling the modified image generation tool based on the modification instruction to obtain a modified image.

[0130] Optionally, after obtaining the modified image, the evaluation tool corresponding to the problem of using the modification can be further detected, if the problem corresponding to the evaluation target image has been solved, the process ends; otherwise, the modification continues, if the problem is still not solved after multiple modifications, the process needs to return Figure 2 to the text-to-image reasoning model for re-image generation.

[0131] The image generation method of the text-to-image reasoning model of the embodiment can not only provide a rich image generation process, but also provide a rich and comprehensive evaluation process, thereby effectively improving the accuracy and high fidelity of the generated target image. In the embodiment, the intermediate results such as the evaluation problem, the evaluation result, and the image generation evaluation result can be effectively displayed, so that the user can understand the evaluation process in real time and improve the user experience.

[0132] The technical solution of the above-mentioned embodiment of the disclosure can solve the creation requirements of complex scenes, such as creating picture book stories, IP character multi-perspective setting images, science fiction video split images and concept setting images, and emotional marketing image strategy series design scenes, and has a very wide application range. Moreover, the technical solution of the embodiment can effectively improve the accuracy and high fidelity of the generated target image due to detailed reasoning and planning.

[0133] The text-to-image reasoning model of the disclosure can be trained in the following stages:

[0134] The first stage is supervised fine-tuning (SFT). The main goal of this stage is to realize the reasoning thinking of the text-to-image reasoning model in the cross-modal creation process and the planning ability of the third-party retrieval tool through SFT, so that the creation intelligent agent reasoning model can reasonably plan whether to retrieve the reference image information of the key subject or picture style according to the user requirement.

[0135] The first phase of the SFT dataset construction: First, based on the capabilities of actual business scenarios and the existing reasoning model that can generate images, we introduced a reasoning and creative thinking process for different image generation requirements. In the data construction of this phase, we focused on distinguishing what kind of requirements need to trigger subject retrieval enhancement or style retrieval enhancement. The scale of collected data can reach 10,000.

[0136] Phase 2, SFT: Building on the foundation of Phase 1 SFT, this phase focuses on enhancing the model's automatic evaluation capabilities for third-party reference images, determining whether the RAG-recalled images are suitable for current needs, and thus improving overall effectiveness.

[0137] The second stage of SFT dataset construction:

[0138] Dataset 1: Using the image retrieval sandbox environment, we retrieved and recalled real images from the dataset collected in the first phase that actually triggered image retrieval enhancement. These images were then fed into the text-based image inference model, which then judged the quality of the recalled images. Finally, the recalled images and the quality assessment process were combined into the SFT dataset from Phase 1.

[0139] Dataset 2: The query input of the actual raw image requirements of online users is fed into the first-stage SFT model, filtering out sample datasets that trigger subject or style retrieval enhancement. The processing solution of "Dataset 1" is then adopted to obtain additional datasets, bringing the collected data size to 10,000.

[0140] Phase III, SFT: Based on the SFT of Phase II, this phase focuses on improving the planning and reasoning capabilities of the image generation steps for complex scenarios, ensuring the rationality and effectiveness of the image generation process. At the same time, this phase also requires enhancing the monitoring capabilities of the security and risks of image generation needs, and providing risk warnings to users.

[0141] Construction of the SFT dataset in the third phase: The dataset in the third phase mainly includes the requirements of scenarios such as multi-image, multi-round, instruction editing, text-to-image, image-to-image, outpainting, inpainting, and super-resolution, with a collection scale of approximately 100,000.

[0142] Phase 4, SFT: Based on the SFT in Phase 3, this phase focuses on improving the ability to self-reflect on and optimize the planning and reasoning skills of complex drawing steps after the introduction of reflective optimization content.

[0143] The fourth stage of SFT dataset: actual mapping results of the enriched data in stages 1-3 are picture quality inspected and evaluated, and the thinking process and conclusion of the reflection evaluation optimization are given. These evaluation processes are spliced into the original dataset. According to the evaluation results, local retouching or global re-planning of mapping is planned, and about 10w is collected.

[0144] The fifth stage, reinforcement learning (RL): based on the fourth stage of SFT, this stage focuses on two reinforcement learning paradigms, continuous group relative policy optimization (GRPO) and direct preference optimization (DPO), to further improve the model output format, the rationality of the mapping step planning, the accuracy of the tool parameter planning, and the rationality of the reflection optimization.

[0145] Among them, GRPO reinforcement learning focuses on improving the output format, whether the planning path is accurate, and whether the parameter setting is accurate. The Reward Function is designed as follows:

[0146] Format Function: judge whether the output content format meets the expected value. If not, reward = -1, if it meets the expected value, reward = 1;

[0147] Search tool planning accuracy detection: judge whether the picture search enhancement is correctly judged. If not, reward = -1, if correct, reward = 1;

[0148] Rationality check of planning steps: judge whether the calling order of the mapping tool of user demand is reasonable and whether the parameter planning of each tool is accurate. If not, reward = -1, if correct, reward = 1.

[0149] Regarding DPO reinforcement learning: focus on improving the effect of prompt expansion and rewriting in the mapping planning stage.

[0150] Construction of RL dataset in the fifth stage:

[0151] GRPO reinforcement learning: according to the actual input mapping demand Query of online users, the actual creative agent reasoning track is generated through online Rollout inference and agent calling, different Reward target functions are designed, and the agent reasoning effect is optimized and improved comprehensively. The dataset is about 10w.

[0152] DPO preference reinforcement learning: according to the data of GRPO reinforcement learning, the partial planning prompt is rewritten and the content is expanded and differentiated. The positive and negative prompt pairs are constructed, and the original picture prompt in the data set is replaced respectively to realize the effect preference optimization of the picture generation. The data set is about 10w.

[0153] The text-to-picture reasoning model of the embodiment can effectively enhance its reasoning ability and picture generation planning ability, and provide effective support for generating high-precision and high-guarantee target pictures.

[0154] Figure 4 is a schematic diagram according to the fourth embodiment of the present disclosure; the picture generation device 400 of the text-to-picture reasoning model of the embodiment includes:

[0155] The acquisition module 401 is configured to acquire a picture generation request of a user.

[0156] The reasoning module 402 is configured to use the pre-trained text-to-picture reasoning model to reason the picture generation request, and acquire a retrieval instruction used when triggering the retrieval enhancement generation tool.

[0157] The step planning module 403 is configured to use the text-to-picture reasoning model to plan picture generation steps based on the retrieval instruction.

[0158] The picture generation module 404 is configured to generate a target picture based on the picture generation steps.

[0159] The picture generation device 400 of the text-to-picture reasoning model of the embodiment realizes the principle and technical effect of the picture generation of the text-to-picture reasoning model by using the above modules, which is the same as the implementation of the above related method embodiment. For details, please refer to the description of the above related method embodiment, which will not be repeated here.

[0160] Figure 5 is a schematic diagram according to the fifth embodiment of the present disclosure; the picture generation device 500 of the text-to-picture reasoning model of the embodiment is based on the technical solutions of the above Figure 4 The picture generation device 500 of the text-to-picture reasoning model of the embodiment is based on the technical solutions of the above Figure 5 The picture generation device 500 of the text-to-picture reasoning model of the embodiment includes the same name and function modules as described above: the acquisition module 501, the reasoning module 502, the step generation module 503, and the picture generation module 404. Figure 4

[0161] Further, in the embodiment, the reasoning module 502 is configured to:

[0162] use the text-to-picture reasoning model to reason the picture generation request, and acquire picture generation requirements.​

[0163] The text-to-picture reasoning model is used to analyze the picture generation requirement, and when it is determined that the retrieval enhancement generation tool needs to be triggered, the type of the recalled reference picture is requested;

[0164] The text-to-picture reasoning model is used to generate a retrieval instruction based on the picture generation requirement and the type of the reference picture.

[0165] Further optionally, in an embodiment of the present disclosure, the inference module 502 is further configured to:

[0166] The text-to-picture reasoning model is used to analyze the picture generation requirement, and it is determined that the retrieval enhancement generation tool does not need to be triggered;

[0167] The step generation module 503 is configured to:

[0168] The text-to-picture reasoning model is used to plan the picture generation steps based on the base picture and the picture generation requirement.

[0169] Further optionally, in an embodiment of the present disclosure, the step generation module 503 is configured to:

[0170] Based on the retrieval instruction, the retrieval enhancement generation tool is called to obtain the reference picture;

[0171] The text-to-picture reasoning model is used to generate the picture generation steps based on the reference picture and the picture generation requirement.

[0172] Further optionally, in an embodiment of the present disclosure, the step generation module 503 is further configured to:

[0173] The text-to-picture reasoning model is used to detect whether the reference picture meets the requirements based on the reference picture and the retrieval instruction.

[0174] Further optionally, in an embodiment of the present disclosure, the step generation module 503 is further configured to:

[0175] The text-to-picture reasoning model is used to detect whether the relevance of the reference picture and the retrieval instruction meets the requirements;

[0176] The text-to-picture reasoning model is used to detect whether the quality of the reference picture meets the requirements;

[0177] When it is detected and determined that the relevance of the reference picture and the retrieval instruction meets the requirements, and the quality of the reference picture also meets the requirements, it is determined that the reference picture meets the requirements.

[0178] Further optionally, in an embodiment of the present disclosure, the inference module 502 is further configured to:

[0179] If it is determined that the reference picture does not meet the requirement, the text-to-picture reasoning model is used to regenerate the retrieval instruction based on the picture requirement and the type of the reference picture.

[0180] Further optionally, in an embodiment of the present disclosure, the step generation module 503 is configured to:

[0181] The text-to-picture reasoning model is used to determine the identification of a target picture tool to be invoked and parameter information required for invoking the target picture tool based on the reference picture and the picture requirement.

[0182] The text-to-picture reasoning model is used to plan the picture generation step based on the reference picture, the picture requirement, the identification of the target picture tool, and the parameter information required for invoking the target picture tool.

[0183] Further optionally, in an embodiment of the present disclosure, the picture generation module 504 is configured to:

[0184] The prompt word information is generated based on the picture generation step.

[0185] The target picture is generated by invoking the target picture tool based on the prompt word information.

[0186] Further optionally, as shown in Figure 5 In an embodiment of the present disclosure, the text-to-picture reasoning model of the picture generation device 500 further includes:

[0187] The evaluation module 505 is configured to:

[0188] The text-to-picture reasoning model is used to generate an evaluation question based on the target picture and the picture requirement.

[0189] The text-to-picture reasoning model is used to determine the identification of a target evaluation tool based on the evaluation question.

[0190] The text-to-picture reasoning model is used to obtain parameter information required for invoking the target evaluation tool based on the target picture and the picture requirement.

[0191] Further optionally, in an embodiment of the present disclosure, the evaluation module 505 is further configured to:

[0192] The target evaluation tool is invoked for evaluation based on the identification of the target evaluation tool and the parameter information required for invoking the target evaluation tool.

[0193] The evaluation result of the target evaluation tool is obtained.

[0194] The text-to-diagram reasoning model is used to generate a diagram evaluation result based on the evaluation result of the target evaluation tool.

[0195] Further optionally, in an embodiment of the present disclosure, the evaluation module 505 is further configured to:

[0196] The text-to-diagram reasoning model is used to detect whether a new diagram needs to be generated based on the diagram evaluation result.

[0197] When it is determined that a new diagram needs to be generated, the text-to-diagram reasoning model is used to optimize the generation of the diagram based on the reference picture and the diagram requirement.

[0198] Further optionally, in an embodiment of the present disclosure, the diagram device 500 of the text-to-diagram reasoning model further comprises:

[0199] The display module 506 is configured to display at least one of the following steps:

[0200] The reasoning information is displayed.

[0201] The diagram generation step is displayed.

[0202] The target picture is displayed.

[0203] The evaluation question is displayed.

[0204] The evaluation result of the target evaluation tool is displayed.

[0205] The diagram evaluation result is displayed.

[0206] The diagram device 500 of the text-to-diagram reasoning model of the present embodiment realizes the principle and technical effects of the text-to-diagram reasoning model by using the above modules, which are the same as the implementation of the above-mentioned related method embodiments. For details, please refer to the description of the above-mentioned related method embodiments, which will not be repeated here.

[0207] Figure 6 is a schematic diagram according to the sixth embodiment of the present disclosure; the agent 600 of the present embodiment comprises:

[0208] The input module 601 is configured to receive a diagram generation request of a user.

[0209] The processing module 602 is configured to determine a target task based on the input information received by the input module, determine a text-to-diagram reasoning model based on the target task, and execute the text-to-diagram reasoning model by calling the text-to-diagram reasoning model to generate a diagram. Figure 1-Figure 3 The method of any embodiment is used to obtain output information.

[0210] The output module 603 is configured to output the output information obtained by the processing module.

[0211] The intelligent agent 600 of the embodiment implements the principle and technical effects of the graph generation of the text-to-graph reasoning model by using the above modules, and the implementation is the same as that of the above related method embodiments. For details, refer to the description of the above related method embodiments, which will not be repeated here.

[0212] In the technical solution of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0213] According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0214] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the present disclosure described and / or claimed in this document.

[0215] As shown in Figure 7 The device 700 includes a computing unit 701 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 702 or a computer program loaded into a random access memory (RAM) 703 from a storage unit 708. Various programs and data required for the operation of the device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0216] Various components in the device 700 are connected to the I / O interface 705, including an input unit 706, such as a keyboard, a mouse, etc., an output unit 707, such as various types of displays, speakers, etc., a storage unit 708, such as a magnetic disk, an optical disk, etc., and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunications networks.

[0217] The computing unit 701 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs various methods and processes described above, such as the above-described methods of the present disclosure. For example, in some embodiments, the above-described methods of the present disclosure can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded onto the RAM 703 and executed by the computing unit 701, one or more steps of the above-described methods of the present disclosure described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the above-described methods of the present disclosure by any other suitable means, such as by means of firmware.

[0218] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0219] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces a means for implementing the functions / acts specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0220] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0221] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0222] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0223] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0224] It should be understood that the various forms of flow shown above can be used to reorder, add, or delete steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, as long as the desired results of the technology disclosed in the present disclosure can be achieved, which is not limited herein.

[0225] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for generating images based on a text-image reasoning model, comprising: Get the user's raw image request; Using a pre-trained text-based image reasoning model, reasoning on the image request to obtain a search instruction used to trigger the search enhancement generation tool; Using the text-to-graph reasoning model, and based on the search instruction, planning the graph generation steps; Based on the image generation step, a target image is generated.

2. The method according to claim 1, wherein A pre-trained text-image reasoning model is used to reason about the image request and obtain the search instructions used to trigger the search enhancement generation tool, including: Using the raw image reasoning model, reasoning on the raw image request to obtain the raw image requirement; The text-image reasoning model is used to analyze the image requirement and determine the type of reference image to be requested for recall when triggering the retrieval enhancement generation tool; The text-image reasoning model is used to generate the search instruction based on the image requirement and the type of the reference image.

3. The method according to claim 2, wherein: The raw image request includes a basic image and text request information; the method further includes: Using the text-generated image reasoning model, analyzing the generated image requirements, and determining that there is no need to trigger the retrieval enhancement generation tool; The text-to-image reasoning model is used to plan the image generation steps based on the basic image and the image generation requirements.

4. The method according to claim 2, wherein: The text-to-graph reasoning model is used to plan the graph generation steps based on the search instruction, including: Based on the search instruction, calling the search enhancement generation tool to obtain the reference image; The text-based image reasoning model is used to plan the image generation steps based on the reference image and the image generation requirements.

5. The method according to claim 4, wherein The method further comprises: The text-image reasoning model is used to detect whether the reference image meets the requirements based on the reference image and the search instruction.

6. The method according to claim 5, wherein: Using the text-image reasoning model, based on the reference image and the search instruction, detecting whether the reference image meets the requirements includes: Using the text-image reasoning model, detecting whether the relevance between the reference image and the search instruction meets the requirements; Using the cultural image reasoning model, detecting whether the quality of the reference image meets the requirements; When it is detected and determined that the relevance between the reference picture and the search instruction meets the requirement, and the quality of the reference picture also meets the requirement, it is determined that the reference picture meets the requirement.

7. The method according to claim 6, wherein: The method further comprises: If it is determined that the reference image does not meet the requirements, the text-generated image reasoning model is used to regenerate the search instruction based on the raw image requirements and the type of the reference image.

8. The method according to claim 4, wherein The text-to-image reasoning model is used to plan the image generation steps based on the reference image and the image generation requirements, including: Using the text-to-image reasoning model, based on the reference image and the image generation requirement, determining an identifier of a target image generation tool to be called and parameter information required to call the target image generation tool; The text-to-image reasoning model is used to plan the image generation steps based on the reference image, the image generation requirement, the identifier of the target image generation tool, and the parameter information required to call the target image generation tool.

9. The method according to claim 8, wherein Based on the image generation step, generating a target image includes: Based on the image generation step, generating prompt word information; Based on the prompt word information, the target image generation tool is called to generate the target image.

10. The method according to any one of claims 1 to 9, wherein: The method further comprises: Using the text-image reasoning model, generating evaluation questions based on the target image and the image requirement; Determining the identity of a target assessment tool based on the assessment question using the text-graph reasoning model; Based on the target image and the raw image requirement, the text-raw image reasoning model is adopted to obtain parameter information for calling the target evaluation tool.

11. The method according to claim 10, wherein: The method further comprises: Based on the identifier of the target evaluation tool and the parameter information for calling the target evaluation tool, calling the target evaluation tool to perform evaluation; Obtaining an evaluation result of the target evaluation tool; The raw graph reasoning model is used to generate raw graph evaluation results based on the evaluation results of the target evaluation tool.

12. The method according to claim 11, wherein The method further comprises: Using the text-generated image reasoning model, based on the image evaluation results, detecting whether the image needs to be regenerated; When it is determined that the image needs to be regenerated, the text-generated image reasoning model is used to optimize the generation steps of the generated image based on the reference image and the generated image requirement.

13. The method according to claim 10, wherein: The method further comprises at least one of the following steps: displaying the retrieval instruction; Demonstrate the steps of generating the image; displaying the target image; Present the assessment questions; Present the evaluation results of the target evaluation tool; as well as The raw image evaluation results are displayed.

14. A device for generating images based on a text-image reasoning model, comprising: The acquisition module is used to obtain the user's raw image request; An inference module, configured to use a pre-trained text-based image inference model to reason on the image request and obtain a search instruction used to trigger the search enhancement generation tool; a step planning module, configured to plan the steps of generating a graph based on the retrieval instruction by using the text-to-graph reasoning model; The image generation module is used to generate a target image based on the image generation step.

15. An intelligent agent comprising: Input module, used to receive user's request for raw image; a processing module, configured to determine a target task based on the input information received by the input module, determine a text-graph reasoning model based on the target task, and execute the method according to any one of claims 1 to 13 by invoking the text-graph reasoning model to obtain output information; An output module is used to output the output information obtained by the processing module.

16. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 13.

17. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

18. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 13.

Images