Image generation method, apparatus and system, and electronic device and computer-readable medium

Abstract

Disclosed in one or more embodiments of the present disclosure are an image generation method, apparatus and system, and an electronic device and a computer-readable medium. The method comprises: after target text is acquired, selecting, from among at least two candidate core content extraction algorithms, a target core content extraction algorithm which matches the target text; then, performing core content extraction processing on the target text by using the target core content extraction algorithm, so as to obtain core content; next, selecting, from among at least two candidate generation models, a target generation model which matches the core content; and finally, performing image generation processing on the core content by using the target generation model, so as to obtain image data corresponding to the target text. In this way, the aim of generating image data from text can be achieved.

Description

Image generation method, device, system, electronic device, and computer-readable medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on September 15, 2023, with application number 202311196002.8 and invention name “Image generation method, device, system, electronic device, computer-readable medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] Embodiments of the present disclosure relate to the field of data processing technology, and in particular, to an image generation method, apparatus, system, electronic device, and computer-readable medium. Background Art

[0004] In some application scenarios (e.g., image generation scenarios, etc.), it may be necessary to perform image generation processing based on the text content provided by the user (e.g., the text content of "a black puppy running") so that the generated image data can represent the semantic information expressed by the text content.

[0005] However, how to realize the requirements described in the previous paragraph is a technical problem that needs to be solved urgently.

[0006] Summary of the Invention

[0007] One or more embodiments of the present disclosure provide an image generation method, apparatus, system, electronic device, and computer-readable medium, which can achieve the purpose of generating image data from text.

[0008] In order to achieve the above objectives, the technical solutions provided by the present disclosure are as follows:

[0009] An embodiment of the present disclosure provides an image generation method, the method comprising:

[0010] Get the target text;

[0011] Selecting a target core content extraction algorithm that matches the target text from at least two candidate core content extraction algorithms;

[0012] Performing core content extraction processing on the target text using the target core content extraction algorithm to obtain the core content;

[0013] Selecting a target generation model that matches the core content from at least two candidate generation models;

[0014] The target generation model is used to perform image generation processing on the core content to obtain image data corresponding to the target text.

[0015] In one possible implementation, the process of determining the target core content extraction algorithm includes:

[0016] Determining a target pattern that matches the target text from the extracted text patterns corresponding to the at least two candidate core content extraction algorithms based on the matching degree representation data between the target text and the extracted text patterns corresponding to the candidate core content extraction algorithms;

[0017] The candidate core content extraction algorithm corresponding to the target style is determined as the target core content extraction algorithm.

[0018] In one possible implementation, selecting a target generation model that matches the core content from at least two candidate generation models includes:

[0019] Based on the similarity between the core content and at least two text samples, and the model performance characterization data of the at least two candidate generation models, the target generation model is determined from the at least two candidate generation models; wherein, for any candidate generation model, the model performance characterization data of the candidate generation model is used to describe the image generation performance presented by the candidate generation model on each of the text samples.

[0020] In one possible implementation, the process of determining the target generation model includes:

[0021] selecting at least one similar sample corresponding to the core content from the at least two text samples based on the similarity between the core content and each of the text samples;

[0022] For any similar sample, selecting at least one performance matching model corresponding to the similar sample from the at least two candidate generation models based on the model performance characterization data of the at least two candidate generation models;

[0023] The target generation model is determined according to at least one performance matching model corresponding to the at least one similar sample.

[0024] In one possible implementation, determining the target generation model based on at least one performance matching model corresponding to the at least one similar sample includes:

[0025] The target generation model is selected from the at least one performance matching model corresponding to the at least one similar sample based on the model performance characterization data of the at least one performance matching model corresponding to the at least one similar sample.

[0026] In one possible implementation, selecting the target generation model from the at least one performance matching model corresponding to the at least one similar sample based on the model performance characterization data of the at least one performance matching model corresponding to the at least one similar sample includes:

[0027] Performing statistical analysis on at least one performance matching model corresponding to the at least one similar sample to obtain occurrence frequency representation data of at least one model to be used; the at least one performance matching model corresponding to the at least one similar sample includes the at least one model to be used;

[0028] For any model to be used, determining similar sample processing performance data of the model to be used based on the model performance characterization data of the model to be used, so that the similar sample processing performance data of the model to be used is used to describe the image generation performance presented by the model to be used on each of the similar samples;

[0029] The target generation model is selected from the at least one model to be used based on the occurrence frequency characterization data of the at least one model to be used and the similar sample processing performance data of the at least one model to be used.

[0030] In one possible implementation, the at least two text samples include I text samples, where I is a positive integer; the at least two candidate generation models include J candidate generation models, where J is a positive integer;

[0031] The process of determining the model performance characterization data of the j-th candidate generation model includes:

[0032] Perform image generation processing on the ith text sample using the jth candidate generation model to obtain the jth generated image corresponding to the ith text sample; j is a positive integer, j≤J, and i is a positive integer, i≤I;

[0033] Determine, based on the j-th generated image corresponding to the i-th text sample, the i-th sample processing performance characterization data corresponding to the j-th candidate generation model; the i-th sample processing performance characterization data corresponding to the j-th candidate generation model is used to represent the image generation performance presented by the j-th candidate generation model on the i-th text sample; j is a positive integer, j≤J, and i is a positive integer, i≤I;

[0034] The model performance characterization data of the j-th candidate generation model is determined based on the first sample processing performance characterization data corresponding to the j-th candidate generation model to the I-th sample processing performance characterization data corresponding to the j-th candidate generation model; j is a positive integer, j≤J.

[0035] In one possible implementation, the i-th sample processing performance characterization data corresponding to the j-th candidate generation model is determined based on the matching degree characterization data between the i-th text sample and the j-th generated image corresponding to the i-th text sample, and the image quality characterization data of the j-th generated image corresponding to the i-th text sample.

[0036] In one possible implementation, before determining the target generative model from the at least two candidate generative models based on the similarity between the core content and the at least two text samples and the model performance characterization data of the at least two candidate generative models, the method further includes:

[0037] Selecting at least one matching generation model corresponding to the core content from the at least two candidate generation models based on the model description information of the at least two candidate generation models;

[0038] The determining the target generation model from the at least two candidate generation models based on the similarity between the core content and the at least two text samples and the model performance characterization data of the at least two candidate generation models includes:

[0039] The target generation model is determined from the at least one matching generation model based on the similarity between the core content and the at least two text samples and the model performance characterization data of the at least one matching generation model.

[0040] In one possible implementation, after selecting a target generation model that matches the core content from at least two candidate generation models, the method further includes:

[0041] updating the core content according to the text format constraint information corresponding to the target generation model to obtain an updated text;

[0042] The using the target generation model to perform image generation processing on the core content to obtain image data corresponding to the target text includes:

[0043] The target generation model is used to perform image generation processing on the updated text to obtain image data corresponding to the target text.

[0044] In a possible implementation manner, the text format constraint information corresponding to the target generation model includes a processed text sample corresponding to the target generation model.

[0045] An embodiment of the present disclosure provides an image generating device, comprising:

[0046] A text acquisition unit, used for acquiring target text;

[0047] A first selection unit is configured to select a target core content extraction algorithm that matches the target text from at least two candidate core content extraction algorithms;

[0048] A core content extraction unit, configured to perform core content extraction processing on the target text using the target core content extraction algorithm to obtain core content;

[0049] A second selection unit is configured to select a target generation model that matches the core content from at least two candidate generation models;

[0050] An image generation unit is used to perform image generation processing on the core content using the target generation model to obtain image data corresponding to the target text.

[0051] An embodiment of the present disclosure provides an image generation system, the image generation system comprising a controller and at least two candidate generation models;

[0052] The controller is used to obtain the target text;

[0053] The controller is further configured to select a target core content extraction algorithm that matches the target text from at least two candidate core content extraction algorithms;

[0054] The controller is further configured to perform core content extraction processing on the target text using the target core content extraction algorithm to obtain the core content;

[0055] The controller is further configured to select a target generation model that matches the core content from at least two candidate generation models;

[0056] The controller is further used to call the target generation model to obtain image data corresponding to the target text; the target generation model is used to perform image generation processing on the core content to obtain image data corresponding to the target text.

[0057] In one possible implementation, the controller is a language model.

[0058] An embodiment of the present disclosure provides an electronic device, the device comprising: a processor and a memory;

[0059] The memory is used to store instructions or computer programs;

[0060] The processor is configured to execute the instructions or computer program in the memory, so that the electronic device executes the image generation method provided by the embodiment of the present disclosure.

[0061] An embodiment of the present disclosure provides a computer-readable medium storing instructions or a computer program. When the instructions or the computer program are executed on a device, the device executes the image generation method provided by the embodiment of the present disclosure.

[0062] An embodiment of the present disclosure provides a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the image generation method provided by the embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0064] FIG1 is a flowchart of an image generation method provided by one embodiment of the present disclosure;

[0065] FIG2 is a schematic diagram of an image generation process provided by an embodiment of the present disclosure;

[0066] FIG3 is a schematic structural diagram of an image generating device provided by an embodiment of the present disclosure;

[0067] FIG4 is a schematic structural diagram of an image generation system provided by an embodiment of the present disclosure;

[0068] FIG5 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0069] In order to enable those skilled in the art to better understand the solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0070] Compared with the related art, one or more embodiments of the present disclosure have at least the following advantages:

[0071] In the technical solution provided by one or more embodiments of the present disclosure, after obtaining the target text, a target core content extraction algorithm that matches the target text is first selected from at least two candidate core content extraction algorithms, so that the target core content extraction algorithm can represent the core content extraction algorithm that exists in the at least two candidate core content extraction algorithms and is suitable for performing core content extraction processing on the target text; then, the target core content extraction algorithm is used to perform core content extraction processing on the target text to obtain core content, so that the core content can represent the core semantic information expressed by the target text; then, a target generation model that matches the core content is selected from at least two candidate generation models, so that the target generation model can represent the model that exists in the at least two candidate generation models and is suitable for performing image generation processing on the target text; finally, the target generation model is used to perform image generation processing on the core content to obtain image data corresponding to the target text, thereby achieving the purpose of generating image data from text.

[0072] In addition, since the core content of the above text can represent the core content of the above target text, the defects caused by interference from non-core content in the target text (for example, content with little or no use value, etc.) can be effectively avoided in the related processes implemented based on the core content (for example, the generation model selection process + image generation process), which is conducive to improving the image generation effect.

[0073] In addition, since the core content above can accurately express the core content expressed by the target text (for example, core semantic information, etc.) with as few characters as possible, the related processes implemented based on the core content (for example, the generation model selection process + image generation process) can consume less time, which is conducive to improving the image generation efficiency.

[0074] In addition, since the target core content extraction algorithm mentioned above refers to a core content extraction algorithm selected from at least two candidate core content extraction algorithms mentioned above and is suitable for performing core content extraction processing on the target text, the target core content extraction algorithm has a high degree of adaptability to the target text, so that the core content subsequently extracted from the target text using the target core content extraction algorithm can more accurately represent the core semantic information expressed by the target text, which is conducive to improving the accuracy of core content extraction, thereby improving the image generation effect.

[0075] To better understand the technical solutions provided by one or more embodiments of the present disclosure, the image generation method provided by one or more embodiments of the present disclosure is described below with reference to some figures. As shown in Figure 1, the image generation method provided by one embodiment of the present disclosure includes S1-S5. Figure 1 is a flowchart of an image generation method provided by one embodiment of the present disclosure.

[0076] S1: Get the target text.

[0077] Among them, the target text refers to the text required as a basis for image generation processing; and the present disclosure does not limit the target text. For example, in some application scenarios, the target text may refer to the text content input by the user with the help of certain input devices (for example, the text content "Generate an image of a dog" shown in Figure 2).

[0078] In addition, the present disclosure does not limit the method for obtaining the target text. For example, in some application scenarios, the target text can be obtained from information input by the user. In another example, in some application scenarios, the target text can be read from some storage space. In another example, in some application scenarios, the target text can refer to text content with image generation processing requirements involved in the execution of other tasks.

[0079] S2: Select a target core content extraction algorithm that matches the target text from at least two candidate core content extraction algorithms.

[0080] For any one of the at least two candidate core content extraction algorithms mentioned above, the candidate core content extraction algorithm is used to perform core content extraction processing on a text content.

[0081] In addition, for at least two candidate core content extraction algorithms above (for example, extraction algorithm 1 to extraction algorithm N shown in Figure 2, N is a positive integer), different candidate core content extraction algorithms have different working principles. For example, in some application scenarios, the at least two candidate core content extraction algorithms may include a prompt core content extraction algorithm, an instruction core content extraction algorithm, an inspiration core content extraction algorithm, and a hypothesis core content extraction algorithm. Among them, the prompt core content extraction algorithm is more suitable for performing core content extraction processing for the text content expressed in accordance with the prompt style (for example, the text content of "a dog"), and the working principle of the prompt core content extraction algorithm can be specifically as follows: directly treating the entire text as the core content (for example, the core content of "a dog"). The instruction core content extraction algorithm is more suitable for performing core content extraction processing for the text content expressed in accordance with the instruction style (for example, the text content of "generating an image of a dog"), and the working principle of the instruction core content extraction algorithm can be specifically as follows: extracting the core part behind the instruction as the core content (for example, the core content of "an image of a dog"). The inspiration core content extraction algorithm is more suitable for performing core content extraction processing on text content expressed in an inspiration style (for example, the text content of "I want to go to the beach"), and the working principle of the inspiration core content extraction algorithm can be specifically: extracting the target subject of desire as the core content (for example, the core content of "beach"). The hypothesis core content extraction algorithm is more suitable for performing core content extraction processing on text content expressed in a hypothesis style (for example, the text content of "If you give me a bouquet of flowers, I will smile happily"), and the working principle of the hypothesis core content extraction algorithm can be specifically: extracting the hypothesis condition and the object of the action to be collected as the core content (for example, the core content of "a bouquet of flowers and a smiling face").

[0082] The target core content extraction algorithm refers to an algorithm that exists in the at least two candidate core content extraction algorithms mentioned above and is suitable for performing core content extraction processing on the target text.

[0083] In addition, the present disclosure does not limit the determination process of the target core content extraction algorithm mentioned above. For example, it can be implemented by any existing or future method that can select the algorithm that best matches a text content from multiple core content extraction algorithms. For another example, the determination process of the target core content extraction algorithm can be implemented with the help of pre-set core content extraction algorithm selection rules. Among them, the core content extraction algorithm selection rules can be set according to the actual application scenario.

[0084] In fact, in some application scenarios, for at least two candidate core content extraction algorithms mentioned above (for example, extraction algorithm 1 to extraction algorithm N shown in Figure 2, where N is a positive integer), different candidate core content extraction algorithms are suitable for performing core content extraction processing on texts of different styles, so that the at least two candidate core content extraction algorithms can be suitable for performing core content extraction processing on texts of multiple styles (for example, text content expressed in a prompt style, text content expressed in an instruction style, text content expressed in an inspiration style, or text content expressed in a hypothesis style, etc.), which is conducive to improving the accuracy of core content extraction.

[0085] Based on the content of the above paragraph, it can be seen that in order to better improve the core content extraction effect, the present disclosure also provides a possible implementation method of the determination process of the above target core content extraction algorithm. Under this implementation method, when different candidate core content extraction algorithms among the at least two candidate core content extraction algorithms above are suitable for core content extraction processing for texts of different styles, the determination process of the target core content extraction algorithm may include the following steps 11-12.

[0086] Step 11: Based on the matching degree representation data between the target text and the extracted text styles corresponding to each candidate core content extraction algorithm, a target style matching the target text is determined from the extracted text styles corresponding to the at least two candidate core content extraction algorithms.

[0087] The extracted text style corresponding to the nth candidate core content extraction algorithm is used to indicate the text style for which the nth candidate core content extraction algorithm is suitable for performing core content extraction processing. Here, n is a positive integer, n≤N, and N is a positive integer representing the number of algorithms in the at least two candidate core content extraction algorithms.

[0088] In addition, for the nth candidate core content extraction algorithm, if the nth candidate core content extraction algorithm is suitable for performing core content extraction processing on the text content of the nth style, then the extracted text style corresponding to the nth candidate core content extraction algorithm is the nth style (for example, a prompt style, an instruction style, an inspiration style, or a hypothesis style, etc.). It should be noted that the present disclosure does not limit the implementation method of the above nth style. For example, it can be set in advance according to the actual application scenario. Wherein, n is a positive integer, n≤N, and N is a positive integer.

[0089] In addition, for the nth candidate core content extraction algorithm, the above target text and the extracted text style corresponding to the nth candidate core content extraction algorithm are used to represent the matching degree between the target text and the extracted text style corresponding to the nth candidate core content extraction algorithm, so that the "matching degree representation data between the target text and the extracted text style corresponding to the nth candidate core content extraction algorithm" can represent whether the text style used by the target text is consistent with the extracted text style corresponding to the nth candidate core content extraction algorithm, and thus the "matching degree representation data between the target text and the extracted text style corresponding to the nth candidate core content extraction algorithm" can represent whether the nth candidate core content extraction algorithm is suitable for performing core content extraction processing on the target text, so that the "matching degree representation data between the target text and the extracted text style corresponding to the nth candidate core content extraction algorithm" can represent the degree of adaptation between the nth candidate core content extraction algorithm and the target text. Wherein, n is a positive integer, n≤N, and N is a positive integer.

[0090] In addition, the present disclosure does not limit the method of obtaining the "data representing the degree of matching between the target text and the extracted text style corresponding to the nth candidate core content extraction algorithm" in the previous paragraph. For example, it can be implemented with the help of a pre-built machine learning model (such as a language model, etc.) that can evaluate the degree of matching between a text content and a text style.

[0091] The target style refers to a style that exists in the extracted text styles corresponding to the at least two candidate core content extraction algorithms and matches the target text, so that the target style can represent the text style that the target text conforms to.

[0092] In addition, the present disclosure does not limit the determination process of the above target style. For example, it can be specifically as follows: after obtaining the target text, determine whether the matching degree representation data between the target text and the extracted text style corresponding to the nth candidate core content extraction algorithm meets the preset style selection condition. If so, the extracted text style corresponding to the nth candidate core content extraction algorithm is determined as the target style, so that the target text can be subsequently processed with the nth candidate core content extraction algorithm corresponding to the target style. Among them, the preset style selection condition can be set in advance according to the application scenario. For example, the preset style selection condition can be: the matching degree representation data between the target text and the target style is not lower than the preset matching degree threshold. n is a positive integer, n≤N, and N is a positive integer.

[0093] For example, in order to better improve the efficiency of core content extraction, the present disclosure also provides a possible implementation method of the above-mentioned target style determination process. Under this implementation method, the target style determination process can be specifically as follows: after obtaining the target text, first sort the N candidate core content extraction algorithms in order from high to low matching degree based on the matching degree representation data between the target text and the extracted text style corresponding to the first candidate core content extraction algorithm, the matching degree representation data between the target text and the extracted text style corresponding to the second candidate core content extraction algorithm, ... (and so on), and the matching degree representation data between the target text and the extracted text style corresponding to the Nth candidate core content extraction algorithm, and obtain the sorting result; then take the candidate core content extraction algorithm that ranks first in the sorting result as the target style that matches the target text.

[0094] Based on the relevant content of step 11 above, it can be known that after obtaining the target text, a target style that matches the above target text can be determined from the extracted text styles corresponding to some pre-specified candidate core content extraction algorithms with better core content extraction performance, so that the candidate core content extraction algorithm corresponding to the target style can be used to perform core content extraction processing on the target text.

[0095] Step 12: Determine the candidate core content extraction algorithm corresponding to the target style as the target core content extraction algorithm, so that the target core content extraction algorithm is used to represent an algorithm suitable for performing core content extraction processing on the text content expressed according to the target style.

[0096] In the present disclosure, for the above target style, if the target style is a prompt style, it can be determined that the core content extraction algorithm corresponding to the target style is the prompt core content extraction algorithm, so the prompt core content extraction algorithm can be determined as the target core content extraction algorithm; if the target style is an instruction style, it can be determined that the core content extraction algorithm corresponding to the target style is the instruction core content extraction algorithm, so the instruction core content extraction algorithm can be determined as the target core content extraction algorithm; if the target style is an inspiration style, it can be determined that the core content extraction algorithm corresponding to the target style is the inspiration core content extraction algorithm, so the inspiration core content extraction algorithm can be determined as the target core content extraction algorithm; if the target style is a hypothesis style, it can be determined that the core content extraction algorithm corresponding to the target style is the hypothesis core content extraction algorithm, so the hypothesis core content extraction algorithm can be determined as the target core content extraction algorithm.

[0097] Based on the relevant content of steps 11 to 12 above, it can be seen that for some application scenarios, after obtaining the text content input by the user (for example, the target text above), it is possible to first determine which algorithm corresponds to the extracted text style of some candidate core content extraction algorithms with better core content extraction performance that the text style presented by the text content is closer to, so that after determining that the text style presented by the text content is closest to the target style, the candidate core content extraction algorithm corresponding to the target style can be directly determined as the target core content extraction algorithm, so that the target core content extraction algorithm can be used to perform core content extraction processing on the text content in the future. This is conducive to eliminating the impact of noise text on the image generation process, thereby facilitating the generation of high-quality images.

[0098] Based on the relevant content of S2 above, it can be known that in some application scenarios, after obtaining the text content input by the user (for example, the text content of "Generate an image of a dog" shown in Figure 2), a target core content extraction algorithm that matches the target text is selected from at least two candidate core content extraction algorithms, so that the target core content extraction algorithm can represent the core content extraction algorithm that exists in the at least two candidate core content extraction algorithms and is more suitable for performing core content extraction processing on the target text, so that the target core content extraction algorithm can be used to perform core content extraction processing on the target text subsequently. This is conducive to eliminating the impact of noise text on the subsequent image generation process, thereby facilitating the generation of high-quality images.

[0099] S3: Use the target core content extraction algorithm to extract the core content of the target text to obtain the core content.

[0100] The core content is used to represent the core semantic information (i.e., core content) described in the target text. For example, when the target text is the text "Generate an image of a dog" as shown in FIG2, the core content may be the core content "image of a dog."

[0101] Based on the relevant content of S3 above, it can be seen that for the target text above, after selecting a target core content extraction algorithm that matches the target text from at least two candidate core content extraction algorithms, the target core content extraction algorithm can be directly used to perform core content extraction processing on the target text to obtain the core content, so that the core content can accurately represent the core semantic information expressed by the target text, thereby enabling the core content to represent the core information carried by the text content, which is conducive to eliminating the impact of noise text on the image generation process, thereby facilitating the generation of high-quality images.

[0102] S4: Select a target generation model that matches the core content from at least two candidate generation models.

[0103] The candidate generation model refers to a pre-built image generation model with good image generation performance. Therefore, any one of the "at least two candidate generation models" mentioned above can be used to perform image generation processing on the input data of the candidate generation model.

[0104] In addition, in some application scenarios, for the above "at least two candidate generation models", different candidate generation models are suitable for implementing image generation processing in different application scenarios, so that different candidate generation models are suitable for image generation processing for different types of text.

[0105] In addition, the present disclosure does not limit the implementation method of the above candidate generation model. For example, in some application scenarios, the candidate generation model can be implemented using a diffusion model (for example, the diffusion model shown in FIG2 ).

[0106] The target generation model refers to a model that exists in the at least two candidate generation models above and is suitable for performing image generation processing on the core content of the target text above, so that the target generation model can be used to perform image generation processing on the core content in the future.

[0107] In addition, the present disclosure does not limit the determination process of the above target generation model. For example, it can be implemented by any existing or future method that can select a model suitable for image generation processing for the above core content from multiple candidate generation models.

[0108] In fact, in order to better improve the image generation effect, the present disclosure also provides a possible implementation method of the target generation model determination process mentioned above. Under this implementation method, the target generation model determination process can be specifically as follows: after obtaining the core content of the target text mentioned above, based on the similarity between the core content and at least two text samples, and the model performance characterization data of the at least two candidate generation models mentioned above, the target generation model is determined from the at least two candidate generation models. Wherein, for any candidate generation model, the model performance characterization data of the candidate generation model is used to describe the image generation performance presented by the candidate generation model on each text sample.

[0109] A text sample refers to the text content required to measure the model performance of an image generation model; and the present disclosure does not limit the implementation method of the above "at least two text samples". For example, in some application scenarios, in order to improve the image generation effect as much as possible, the "at least two text samples" may include a large amount of text content, so that the "at least two text samples" can cover as many text contents as possible in application fields that may have image generation requirements. In addition, the present disclosure does not limit the method of obtaining the "at least two text samples". For example, the "at least two text samples" can be manually input by relevant personnel. For another example, the "at least two text samples" can be obtained from the training data of the image generation model in various fields.

[0110] In addition, for the above at least two text samples, when the at least two text samples include I text samples, the similarity between the above core content and the i-th text sample is used to represent the similarity between the core content and the i-th text sample in at least one aspect (for example, semantic aspect, etc.); and the present disclosure does not limit the implementation method of the "similarity between the core content and the i-th text sample", for example, in some application scenarios, the "similarity between the core content and the i-th text sample" may include the semantic similarity between the core content and the i-th text sample. Furthermore, the present disclosure does not limit the method for obtaining the "similarity between the core content and the i-th text sample", for example, it can be implemented using any existing or future method that can measure the similarity between two text contents. Wherein, i is a positive integer, i≤I, I is a positive integer, and I represents the number of text samples in the at least two text samples.

[0111] For the at least two candidate generation models mentioned above, when the at least two candidate generation models include J candidate generation models, the model performance characterization data of the jth candidate generation model is used to characterize the performance of the jth candidate generation model; and the present disclosure does not limit the implementation method of the "model performance characterization data of the jth candidate generation model". For example, in some application scenarios, the "model performance characterization data of the jth candidate generation model" can be implemented using any existing or future method that can measure the model performance of a model. For another example, in some application scenarios, in order to better improve the performance characterization effect, the "model performance characterization data of the jth candidate generation model" can be used to describe the image generation performance presented by the jth candidate generation model on each text sample, so that the "model performance characterization data of the jth candidate generation model" can represent the degree of adaptation between the jth candidate generation model and each text sample, thereby enabling the "model performance characterization data of the jth candidate generation model" to represent which text samples the jth candidate generation model is suitable for processing and which text samples it is not suitable for processing. Wherein, j is a positive integer, j≤J, J is a positive integer, and J represents the number of models in the at least two candidate generation models.

[0112] In addition, the present disclosure does not limit the method of obtaining the "model performance characterization data of the jth candidate generation model" in the previous paragraph. For example, in some application scenarios, in order to better improve the performance characterization effect, the "model performance characterization data of the jth candidate generation model" may include the following steps 21-23.

[0113] Step 21: Use the j-th candidate generation model above to perform image generation processing on the i-th text sample to obtain the j-th generated image corresponding to the i-th text sample; i is a positive integer, i≤I.

[0114] Among them, the j-th generated image corresponding to the i-th text sample refers to the image data obtained by performing image generation processing on the i-th text sample using the j-th candidate generation model above, and i is a positive integer, i≤I.

[0115] Step 22: Based on the j-th generated image corresponding to the i-th text sample above, determine the i-th sample processing performance characterization data corresponding to the j-th candidate generation model above; the i-th sample processing performance characterization data corresponding to the j-th candidate generation model is used to represent the image generation performance presented by the j-th candidate generation model on the i-th text sample; i is a positive integer, i≤I.

[0116] Among them, the i-th sample processing performance characterization data corresponding to the j-th candidate generation model is used to characterize the performance presented by the j-th candidate generation model when performing image generation processing on the i-th text sample, so that the "i-th sample processing performance characterization data corresponding to the j-th candidate generation model" can represent the image generation performance presented by the j-th candidate generation model on the i-th text sample, so that the i-th sample processing performance characterization data can represent the degree of adaptation between the j-th candidate generation model and the i-th text sample, and further enable the i-th sample processing performance characterization data to indicate whether the j-th candidate generation model is suitable for image generation processing on the i-th text sample, i is a positive integer, i≤I.

[0117] In addition, the present disclosure does not limit the method for determining the above-mentioned "the i-th sample processing performance characterization data corresponding to the j-th candidate generation model". For example, it can be implemented using any existing or future method that can measure the image generation performance presented by an image generation model under a certain text.

[0118] For example, in some application scenarios, in order to improve the performance characterization effect, the present disclosure also provides a possible implementation method of the above-mentioned "performance characterization data of sample processing of the i-th candidate generation model corresponding to the j-th candidate generation model". Under this implementation method, the determination process of the "performance characterization data of sample processing of the i-th candidate generation model corresponding to the j-th candidate generation model" can specifically include the following steps 221-223.

[0119] Step 221: Perform analysis and processing on the j-th generated image corresponding to the i-th text sample above under at least one quality dimension to obtain image quality characterization data of the j-th generated image corresponding to the i-th text sample, so that the image quality characterization data can represent the state of the j-th generated image corresponding to the i-th text sample under the at least one quality dimension.

[0120] Among them, at least one quality dimension refers to a pre-set reference dimension required to measure the quality state of a generated image; and the present disclosure does not limit the at least one quality dimension. For example, in some application scenarios, the at least one quality dimension may include aesthetics. For example, in some application scenarios, the at least one quality dimension may also include the rationality of the layout and / or the naturalness of the color transition. Among them, the aesthetics is used to indicate whether the generated image is beautiful. The rationality of the layout is used to indicate whether the layout state in the generated image is contrary to the laws of nature. The naturalness of the color transition is used to indicate whether there is a cliff-like change in color in the generated image.

[0121] The above "image quality characterization data of the jth generated image corresponding to the i-th text sample" is used to characterize the quality state of the j-th generated image corresponding to the i-th text sample, so that the "image quality characterization data of the j-th generated image corresponding to the i-th text sample" can represent the state of the j-th generated image in at least one quality dimension.

[0122] In addition, the present disclosure does not limit the implementation method of the above-mentioned "image quality characterization data of the jth generated image corresponding to the i-th text sample". For example, in some application scenarios, the "image quality characterization data of the jth generated image corresponding to the i-th text sample" may include the aesthetics of the j-th generated image. For another example, in some application scenarios, the "image quality characterization data of the j-th generated image corresponding to the i-th text sample" may include the aesthetics of the j-th generated image, the rationality of the layout of the j-th generated image, and the naturalness of the color transition of the j-th generated image.

[0123] For example, the present disclosure does not limit the method for determining the above-mentioned "image quality characterization data of the jth generated image corresponding to the i-th text sample". For example, in some application scenarios, the determination process of the "image quality characterization data of the jth generated image corresponding to the i-th text sample" can be specifically as follows: first calculate the state characterization data of the j-th generated image corresponding to the i-th text sample under each quality dimension, so that the state characterization data can represent the state of the j-th generated image under the corresponding quality dimension; then determine the image quality characterization data of the j-th generated image corresponding to the i-th text sample based on these state characterization data (for example, directly aggregate these state characterization data to obtain the image quality characterization data of the j-th generated image corresponding to the i-th text sample, so that the image quality characterization data includes these state characterization data; or, directly perform weighted sum processing on these state characterization data to obtain the image quality characterization data of the j-th generated image corresponding to the i-th text sample, etc.).

[0124] Step 222: Calculate the matching degree representation data between the i-th text sample and the j-th generated image corresponding to the i-th text sample, so that the matching degree representation data is used to indicate whether the content described by the j-th generated image corresponding to the i-th text sample is consistent with the semantic content expressed by the i-th text sample.

[0125] It should be noted that the present disclosure does not limit the implementation of the above step 222. For example, it can be implemented by using any existing or future method that can measure the degree of matching between a text and image data generated for the text.

[0126] Step 223: Determine the i-th sample processing performance characterization data corresponding to the j-th candidate generation model based on the matching degree characterization data between the i-th text sample and the j-th generated image corresponding to the i-th text sample, and the image quality characterization data of the j-th generated image corresponding to the i-th text sample.

[0127] It should be noted that the present disclosure does not limit the implementation method of the above step 223. For example, in some application scenarios, the step 223 can specifically be: combining the matching degree characterization data between the above i-th text sample and the j-th generated image corresponding to the i-th text sample, and the image quality characterization data of the j-th generated image corresponding to the i-th text sample, to obtain the i-th sample processing performance characterization data corresponding to the j-th candidate generation model, so that the i-th sample processing performance characterization data includes the matching degree characterization data and the image quality characterization data.

[0128] For example, in some application scenarios, the above step 223 can be specifically performed as follows: weighted summation processing is performed on the matching degree representation data between the above i-th text sample and the j-th generated image corresponding to the i-th text sample, and the image quality representation data of the j-th generated image corresponding to the i-th text sample to obtain the i-th sample processing performance representation data corresponding to the j-th candidate generation model.

[0129] Based on the relevant contents of steps 221 to 223 above, it can be known that in some application scenarios, for the j-th candidate generation model above, the i-th sample processing performance characterization data corresponding to the j-th candidate generation model is determined based on the matching degree characterization data between the i-th text sample and the j-th generated image corresponding to the i-th text sample, and the image quality characterization data of the j-th generated image corresponding to the i-th text sample, so that the i-th sample processing performance characterization data can more accurately represent the image generation performance presented by the j-th candidate generation model on the i-th text sample, thereby enabling the i-th sample processing performance characterization data to more accurately represent the degree of adaptation between the j-th candidate generation model and the i-th text sample, and further enabling the i-th sample processing performance characterization data to more accurately represent whether the j-th candidate generation model is suitable for image generation processing for the i-th text sample.

[0130] Based on the relevant content of step 22 above, it can be known that for the jth candidate generation model above, after obtaining the jth generated image generated by the jth candidate generation model for the i-th text sample, the i-th sample processing performance characterization data corresponding to the j-th candidate generation model can be determined based on the j-th generated image, so that the i-th sample processing performance characterization data is used to represent the image generation performance presented by the j-th candidate generation model on the i-th text sample, so that the model performance of the j-th candidate generation model can be measured based on the i-th sample processing performance characterization data.

[0131] Step 23: Determine the model performance characterization data of the jth candidate generation model based on the 1st sample processing performance characterization data corresponding to the jth candidate generation model to the Ith sample processing performance characterization data corresponding to the jth candidate generation model.

[0132] It should be noted that the present disclosure does not limit the implementation method of the above step 23. For example, in one possible implementation method, the step 23 can be specifically as follows: directly performing collective processing on the first sample processing performance characterization data corresponding to the jth candidate generation model to the first sample processing performance characterization data corresponding to the jth candidate generation model to obtain the model performance characterization data of the jth candidate generation model, so that the model performance characterization data of the jth candidate generation model includes the first sample processing performance characterization data corresponding to the jth candidate generation model to the first sample processing performance characterization data corresponding to the jth candidate generation model, so that the model performance characterization data of the jth candidate generation model can describe the image generation performance presented by the jth candidate generation model on each text sample, and further the model performance characterization data of the jth candidate generation model can represent the degree of adaptation between the jth candidate generation model and each text sample, so that the model performance characterization data of the jth candidate generation model can represent which text samples the jth candidate generation model is suitable for processing and which text samples it is not suitable for processing.

[0133] Based on the relevant content of steps 21 to 23 above, it can be known that in some application scenarios, for the jth candidate generation model above, the image generation performance presented by the jth candidate generation model on each text sample can be calculated first; and then based on these image generation performances, the model performance characterization data of the jth candidate generation model can be determined, so that the model performance characterization data can describe the image generation performance presented by the jth candidate generation model on each text sample, so that the model performance characterization data can represent the degree of adaptation between the jth candidate generation model and each text sample, and then the model performance characterization data can represent what type of text the jth candidate generation model is suitable for image generation processing.

[0134] In addition, the present disclosure does not limit the implementation method of the above step of "after obtaining the core content of the target text, determining the target generation model from the at least two candidate generation models based on the similarity between the core content and at least two text samples, and the model performance characterization data of the at least two candidate generation models". For example, it can be specifically as follows: first, based on the similarity between the core content and at least two text samples, the text sample most similar to the core content is screened out from the at least two text samples; then, based on the model performance characterization data of the at least two candidate generation models, the model that is most suitable for image generation processing for the "text sample most similar to the core content" is screened out from the at least two candidate generation models as the target generation model.

[0135] For example, in some application scenarios, in order to better improve the image generation effect, the present disclosure provides an implementation method of the above-mentioned target generation model determination process. In this implementation method, the target generation model determination process may specifically include the following steps 31 to 33.

[0136] Step 31: After obtaining the core content of the target text, select at least one similar sample corresponding to the core content from the at least two text samples according to the similarity between the core content and each text sample.

[0137] Among them, the similar sample refers to a text sample that exists in the at least two text samples above and whose similarity with the core content of the target text above reaches a preset similarity condition. The preset similarity condition can be pre-set. For example, in some application scenarios, the preset similarity condition can specifically be: the similarity between the similar sample and the core content reaches a preset threshold. For example, in some application scenarios, after the at least two text samples are arranged in order from most similar to least similar based on the similarity between the core content and each text sample, the preset similarity condition can specifically be: the arrangement sequence number corresponding to the similar sample does not exceed the preset sequence number threshold.

[0138] As can be seen, in some application scenarios, for the target text above, after obtaining the core content of the target text, the similarity between the core content and each text sample can be calculated; then, based on these similarities, all text samples are arranged in order from most similar to least similar to obtain the sample ranking corresponding to the core content; then, the first E text samples in the sample ranking are used as the E similar samples corresponding to the core content. Where E is a positive integer and can be pre-set, for example, E = 5.

[0139] Step 32: For any similar sample, based on the model performance characterization data of at least two candidate generation models, select at least one performance matching model corresponding to the similar sample from the at least two candidate generation models.

[0140] The at least one performance matching model corresponding to the e-th similar sample refers to a model among the at least two candidate generation models whose image generation performance on the e-th similar sample meets a preset performance condition, so that the "at least one performance matching model corresponding to the e-th similar sample" can represent a model among the at least two candidate generation models that is relatively suitable for performing image generation processing on the e-th similar sample. The preset performance condition can be pre-set. For example, in some application scenarios, the preset performance condition can specifically be: the image generation performance of the performance matching model on the e-th similar sample meets a preset performance threshold. For another example, in some application scenarios, after the at least two candidate generation models are ranked from highest to lowest image generation performance based on their image generation performance on the e-th similar sample, the preset performance condition can specifically be: the ranking sequence number corresponding to the performance matching model does not exceed a preset sequence number threshold. e is a positive integer, e≤E, E is a positive integer, and E represents the number of samples in the "at least one similar sample" above.

[0141] It can be seen that in some application scenarios, for the target text above, after determining the e-th similar sample that is relatively similar to the core content of the target text from the at least two text samples above, the image generation performance presented by the at least two candidate generation models on the e-th similar sample can be determined based on the model performance characterization data of the at least two candidate generation models above; then, based on the image generation performance presented by the at least two candidate generation models on the e-th similar sample, the at least two candidate generation models are arranged in order from the highest image generation performance to the lowest image generation performance to obtain the model ranking corresponding to the e-th similar sample; then, the top L candidate generation models in the model ranking are used as the L performance matching models corresponding to the e-th similar sample. Wherein, L is a positive integer, and L can be set in advance according to the application scenario, for example, L is 5.

[0142] Step 33: Determine a target generation model based on at least one performance matching model corresponding to at least one similar sample.

[0143] It should be noted that the present disclosure does not limit the implementation method of the above step 33. For example, in some application scenarios, the step 33 can be specifically as follows: at least one performance matching model corresponding to at least one similar sample above is used as a target generation model, so that the quality status of the image data generated by these target generation models for the core content of the above target text can be used to select the image data with the best quality from the image data generated by these target generation models as the image data corresponding to the target text.

[0144] For example, in some application scenarios, step 33 above can be specifically as follows: randomly selecting a model from at least one performance matching model corresponding to at least one similar sample above as the target generation model, so that the target generation model can be directly used to generate image data corresponding to the target text.

[0145] For example, in some application scenarios, in order to better improve the image generation effect, the present disclosure also provides a possible implementation method of step 33 above. In this implementation method, step 33 can specifically be: based on the model performance characterization data of at least one performance matching model corresponding to at least one similar sample above, select a target generation model from at least one performance matching model corresponding to the at least one similar sample, so that the target generation model can present better image generation performance on the core content of the target text above.

[0146] In addition, the present disclosure does not limit the implementation method of the step in the above paragraph of "selecting a target generation model from the at least one performance matching model corresponding to the at least one similar sample based on the model performance characterization data of the at least one performance matching model corresponding to the at least one similar sample". For example, it can be specifically as follows: based on the model performance characterization data of the at least one performance matching model corresponding to the at least one similar sample, the model with the highest comprehensive image generation performance presented on the at least one similar sample is screened out from the at least one performance matching model corresponding to the at least one similar sample as the target generation model. The comprehensive image generation performance is used to represent the comprehensive state of the image generation performance presented by an image generation model on all similar samples; and the present disclosure does not limit the determination process of the comprehensive image generation performance. For example, it can be specifically as follows: performing weighted sum processing on the image generation performance presented by the image generation model on each similar sample to obtain the comprehensive image generation performance presented by the image generation model on the at least one similar sample.

[0147] In addition, in order to better improve the image generation effect, the present disclosure also provides a possible implementation of the above step 33. Under this implementation, the step 33 may specifically include the following steps 331 to 333.

[0148] Step 331: Statistically analyze at least one performance matching model corresponding to the above at least one similar sample to obtain occurrence frequency characterization data of at least one model to be used; the at least one performance matching model corresponding to the at least one similar sample includes the at least one model to be used.

[0149] The "to-be-used model" refers to a model that exists in the at least one performance matching model corresponding to the at least one similar sample and appears at least once, so that the "at least one to-be-used model" is used to represent different models that appear in the at least one performance matching model corresponding to the at least one similar sample. It can be seen that for any to-be-used model in the "at least one to-be-used model", the to-be-used model appears at least once in the at least one performance matching model corresponding to the at least one similar sample, so that the to-be-used model appears at least once in the at least one performance matching model corresponding to the at least one similar sample.

[0150] The frequency characterization data of the f-th model to be used is used to represent the frequency of occurrence of the f-th model to be used in at least one performance matching model corresponding to at least one similar sample above; and the present disclosure does not limit the implementation method of the "frequency characterization data of the f-th model to be used", for example, the "frequency characterization data of the f-th model to be used" may specifically include the number of occurrences of the f-th model to be used in at least one performance matching model corresponding to the at least one similar sample. Wherein, f is a positive integer, f≤F, F is a positive integer, and F represents the number of models in the above "at least one model to be used" (that is, the number of different models appearing in the at least one performance matching model corresponding to the at least one similar sample).

[0151] Step 332: For any model to be used, determine the similar sample processing performance data of the model to be used based on the model performance characterization data of the model to be used, so that the similar sample processing performance data of the model to be used is used to describe the image generation performance presented by the model to be used on each similar sample.

[0152] The similar sample processing performance data of the f-th model to be used is used to describe the image generation performance presented by the f-th model to be used on each similar sample. Moreover, the present disclosure does not limit the implementation method of the similar sample processing performance data of the f-th model to be used. For example, the similar sample processing performance data of the f-th model to be used may include data used to characterize the image generation performance presented by the f-th model to be used on the first similar sample, data used to characterize the image generation performance presented by the f-th model to be used on the second similar sample, ..., data used to characterize the image generation performance presented by the f-th model to be used on the E-th similar sample. f is a positive integer, f≤F, and F is a positive integer.

[0153] Step 333: Select a target generation model from at least one model to be used based on the occurrence frequency characterization data of at least one model to be used and the similar sample processing performance data of at least one model to be used.

[0154] It should be noted that the present disclosure does not limit the implementation method of the above step 333. For example, it can be specifically as follows: based on the frequency characterization data of at least one model to be used and the similar sample processing performance data of at least one model to be used, a target generation model that meets the preset selection conditions is selected from at least one model to be used. The preset selection condition can be set in advance. For example, the preset selection condition can be as follows: the frequency characterization data of the target generation model meets the preset high-frequency condition, and the similar sample processing performance data of the target generation model meets the preset high-performance condition. The preset high-frequency condition can be set in advance according to the application scenario. For example, the preset high-frequency condition can be as follows: the number of occurrences of the target generation model is higher than the preset number threshold. The preset high-performance condition can be set in advance according to the application scenario. For example, the preset high-performance condition can be as follows: the image generation performance presented by the target generation model on each similar sample is higher than the preset high-performance threshold.

[0155] For example, in some application scenarios, the above step 333 can be specifically as follows: first, perform weighted summation on the occurrence frequency representation data of the f-th model to be used and the similar sample processing performance data of the f-th model to be used to obtain the usage probability representation data of the f-th model to be used, so that the usage probability representation data can represent the possibility of using the f-th model to be used to perform image generation processing on the core content of the above target text, f is a positive integer, f≤F, F is a positive integer; then, based on the usage probability representation data of all models to be used, select the model with the largest usage probability representation data from these models to be used as the target generation model.

[0156] For example, in some application scenarios, step 333 may specifically include inputting the frequency representation data of at least one model to be used and the similar sample processing performance data of at least one model to be used into a pre-built language model, so that the language model can select a target generation model with a higher frequency of occurrence and better performance from the at least one model to be used based on these data. The language model can be used to select a target generation model with a higher frequency of occurrence and better performance from the at least one model to be used based on the frequency representation data of the at least one model to be used and the similar sample processing performance data of the at least one model to be used.

[0157] Based on the relevant contents of steps 331 to 333 above, it can be known that for some application scenarios, after obtaining at least one performance matching model corresponding to at least one similar sample for the target text, a model with a relatively high frequency of occurrence and relatively good performance can be selected from these performance matching models as a target model, so that the target model can represent a model suitable for image generation processing for the core content of the target text.

[0158] Based on the relevant contents of steps 31 to 33 above, it can be known that in some application scenarios, for the target text above, after obtaining the core content of the target text above, the similarity between the core content and each text sample can be calculated first, so that similar samples that are relatively similar to the core content can be screened out from these text samples based on these similarities; and then, based on the image generation performance presented by some candidate generation models under each text sample, models that present relatively good image generation performance on each similar sample are selected from these candidate generation models respectively; and then, a model with relatively high occurrence frequency and relatively good performance is selected from these selected models as the target model, so that the target model can represent a model suitable for image generation processing for the core content of the target text.

[0159] In fact, in some application scenarios, in order to better improve the image generation effect, the present disclosure provides an implementation method of the above target generation model determination process. In this implementation method, the target generation model determination process may specifically include the following steps 41-42.

[0160] Step 41: After obtaining the core content of the target text, select at least one matching generation model corresponding to the core content from the at least two candidate generation models based on the model description information of the at least two candidate generation models.

[0161] Wherein, the model description information of the jth candidate generation model is used to describe the characteristics of the jth candidate generation model; and the present disclosure does not limit the implementation method of the model description information of the jth candidate generation model. For example, in one possible implementation method, the model description information of the jth candidate generation model may include at least one model attribute of the jth candidate generation model. Wherein, the at least one model attribute of the jth candidate generation model is used to represent the attribute state of the candidate generation model; and the present disclosure does not limit the at least one model attribute of the jth candidate generation model. For example, in some application scenarios, the at least one model attribute of the jth candidate generation model may include all or part of the attribute labels pre-annotated for the jth candidate generation model. j is a positive integer, j≤J, J is a positive integer, and J represents the number of models in the at least two candidate generation models.

[0162] The above "at least one matching generation model" refers to a model that exists in the at least two candidate generation models above and is relatively matched with the core content of the target text in terms of model description information, so that the "at least one matching generation model" can represent a model obtained from the analysis of the model description information dimension and suitable for image generation processing for the core content of the target text.

[0163] It can be seen that for any one of the matching generation models in the above "at least one matching generation model", the matching representation data between the model description information of the matching generation model and the core content can meet the preset information matching condition. The preset information matching condition can be set in advance. For example, the preset information matching condition can specifically be: the matching representation data between the model description information of the matching generation model and the core content can meet the preset information matching threshold. For another example, when the at least two candidate generation models are sorted in order from high match to low match based on the matching representation data between the model description information of each candidate generation model and the core content, the preset information matching condition can specifically be: the arrangement sequence number corresponding to the matching generation model does not exceed the preset sequence number threshold. Among them, the matching representation data between the model description information of the jth candidate generation model and the core content is used to represent the degree of matching between the model description information of the jth candidate generation model and the core content, so that the "matching representation data between the model description information of the jth candidate generation model and the core content" can represent the degree of adaptation between the jth candidate generation model and the core content, thereby enabling the "matching representation data between the model description information of the jth candidate generation model and the core content" to characterize whether the jth candidate generation model is suitable for image generation processing for the core content. j is a positive integer, j≤J, J is a positive integer, and J represents the number of models in the at least two candidate generation models.

[0164] In addition, the present disclosure does not limit the method for determining the "matching representation data between the model description information of the j-th candidate generation model and the core content" in the previous paragraph. For example, it can be implemented using any existing or future method that can measure whether the two pieces of information match.

[0165] For example, in some application scenarios, in order to improve matching efficiency, when the above at least two candidate generation models are divided into multiple model sets, and different model sets represent different categories, for any model in any model set, the process of determining the matching representation data between the model description information of the model and the core content can be: calculating the degree of matching between the model description information corresponding to the model set to which the model belongs and the core content of the target text above, as the matching representation data between the model description information of the model and the core content. Wherein, for any model set, the model description information corresponding to the model set is used to characterize the characteristics of all models in the model set; and the present disclosure does not limit the implementation method of the model description information corresponding to the model set, for example, the model description information corresponding to the model set can specifically include the model description information of all models in the model set. For another example, the model description information corresponding to the model set can include some information extracted from the model description information of all models in the model set (for example, model category, model common attributes, etc.), so that the model description information corresponding to the model set can represent the common characteristics (for example, category, etc.) presented by all models in the model set under one or more dimensions.

[0166] Based on the content of the above paragraph, it can be seen that in some application scenarios, in order to improve the matching efficiency, the above step 41 can achieve rough matching by means of classification or clustering. Based on this, it can be seen that in a possible implementation method, the above step 41 can be specifically as follows: first, based on the model description information of the at least two candidate generation models above, these at least two candidate generation models are classified (for example, classification processing or clustering processing, etc.) to obtain multiple model sets, so that different model sets are used to record at least one model under different categories, so that the models in the same model set belong to the same category, and the models in different model sets belong to different categories; then, based on the degree of matching between the core content of the target text above and the model description information corresponding to each model set, the model set that best matches the core content is selected from these model sets, and all or part of the models in the best matching model set are used as the matching generation model corresponding to the core content.

[0167] Based on the relevant content of step 41 above, it can be known that in some application scenarios, for the above target text, after obtaining the core content of the above target text, based on the model description information of the above at least two candidate generation models, at least one matching generation model corresponding to the core content is selected from the at least two candidate generation models, so that the at least one matching generation model can represent a model determined based on the model description information and suitable for image generation processing for the core content of the target text.

[0168] Step 42: Determine a target generation model from the at least one matching generation model based on the similarity between the core content and the at least two text samples and the model performance characterization data of the at least one matching generation model.

[0169] It should be noted that the present disclosure does not limit the implementation method of the above step 42. For example, the implementation method of the step 42 is similar to the implementation method of the step described above of "determining the target generation model from the at least two candidate generation models based on the similarity between the core content and the at least two text samples, and the model performance characterization data of the at least two candidate generation models above". For the sake of brevity, it will not be repeated here.

[0170] Based on the relevant contents of steps 41 to 42 above, it can be known that in some application scenarios, for the target text above, after obtaining the core content of the target text above, at least one matching generation model corresponding to the core content can be selected from the at least two candidate generation models based on the model description information of the at least two candidate generation models above, so that the at least one matching generation model can represent a model suitable for image generation processing for the core content of the target text determined based on the model description information, to complete the coarse screening stage; then, based on the similarity between the core content and the at least two text samples, and the model performance characterization data of the at least one matching generation model, the target generation model is determined from the at least one matching generation model, so that the target generation model can represent a model suitable for image generation processing for the core content of the target text, to complete the fine screening stage. It can be seen that because the target generation model is determined based on the relevant information (such as model attributes, model performance, etc.) under multiple dimensions of these candidate generation models, so that the candidate generation model obtained by the final screening is more suitable for image generation processing for the core content of the target text, which is conducive to improving the image generation effect.

[0171] Based on the relevant content of S4 above, it can be known that in some application scenarios, for the target text above, after obtaining the core content of the target text, a target generation model that matches the core content can be selected from some pre-built candidate generation models with good image processing performance, so that the target generation model has a high degree of adaptability with the core content, so that the target generation model can be used to perform image generation processing on the core content in the future.

[0172] S5: Use the target generation model to perform image generation processing on the core content to obtain image data corresponding to the target text.

[0173] In the present disclosure, for the target text described above, after obtaining the core content of the target text and a target generation model that matches the core content, the core content can be directly input into the target generation model so that the target generation model performs image generation processing on the core content to obtain image data corresponding to the target text. In particular, because the target generation model is more suitable for performing image generation processing on the core content, the image data generated by the target generation model for the core content can be more consistent with the semantic information expressed by the core content, which is conducive to improving the image generation effect.

[0174] In fact, in order to better improve the image generation effect, the present disclosure also provides a possible implementation method of the process of determining the image data corresponding to the above target text. Under this implementation method, the process of determining the image data corresponding to the target text can specifically include the following steps 51-52.

[0175] Step 51: After obtaining the core content of the target text and the target generation model matching the core content, the core content is updated according to the text format constraint information corresponding to the target generation model to obtain an updated text.

[0176] The text format constraint information corresponding to the target generation model is used to indicate the text format for which the target generation model is more suitable for image generation processing.

[0177] In addition, the present disclosure does not limit the implementation method of the above-mentioned "text format constraint information corresponding to the target generation model". For example, in some application scenarios, the "text format constraint information corresponding to the target generation model" may include constraint description content under one or more format dimensions. It should be noted that the present disclosure does not limit the implementation method of the one or more format dimensions. For example, it can be set according to the actual application scenario. In addition, the present disclosure does not limit the implementation method of the constraint description content of the one or more format dimensions. For example, it can be set according to the actual application scenario.

[0178] For example, in some application scenarios, the "text format constraint information corresponding to the target generation model" mentioned above may include a processed text sample corresponding to the target generation model. The processed text sample refers to a text example pre-set for the target generation model and suitable for image processing using the target generation model, so that the processed text sample can indicate which text format the target generation model is more suitable for image generation processing.

[0179] In addition, the present disclosure does not limit the number of the above-mentioned processed text samples. For example, in some application scenarios, the above-mentioned "text format constraint information corresponding to the target generation model" may include one or more processed text samples corresponding to the target generation model, so that the "text format constraint information corresponding to the target generation model" can indicate as comprehensively and accurately as possible what format of text the target generation model is more suitable for image generation processing.

[0180] In addition, the present disclosure does not limit the method for obtaining the above-mentioned processed text sample. For example, the processed text sample can be pre-set. For another example, the processed text sample can be automatically determined based on the training data of the above-mentioned target generation model.

[0181] The updated text refers to the text obtained by updating the core content of the above target text according to the text format constraint information corresponding to the above target generation model, so that the semantic information expressed by the updated text is consistent with the semantic information expressed by the core content, and the updated text satisfies the text format described by the text format constraint information, so that the updated text can express the semantic information expressed by the core content on the premise of satisfying the text format described by the text format constraint information, and further enables the updated text to express the semantic information expressed by the core content according to the text format requirements of the target generation model.

[0182] It should be noted that the present disclosure does not limit the implementation method of the update process. For example, in some application scenarios, the update process can be implemented by rewriting. Based on this, it can be seen that in one possible implementation method, the above step 51 can be specifically as follows: after obtaining the core content of the above target text and the target generation model that matches the core content, the core content is rewritten according to the text format constraint information corresponding to the target generation model to obtain an updated text, so that the updated text can represent the text obtained by rewriting the core content of the above target text according to the text format constraint information corresponding to the above target generation model.

[0183] Based on the relevant content of step 51 above, it can be known that for some application scenarios, after obtaining the core content of the above target text and the target generation model that matches the core content, the core content can be updated according to the text format constraint information corresponding to the target generation model to obtain an updated text, so that the updated text can express the semantic information expressed by the core content on the premise of satisfying the text format described by the text format constraint information, thereby enabling the updated text to express the semantic information expressed by the core content in accordance with the text format requirements of the target generation model.

[0184] Step 52: Use the above target generation model to perform image generation processing on the updated text to obtain image data corresponding to the target text.

[0185] In the present disclosure, after obtaining the updated text corresponding to the core content of the target text above, the updated text can be directly input into the target generation model that matches the core content, so that the target generation model can perform image generation processing on the updated text, and obtain and output image data corresponding to the target text. In particular, because the updated text can express the semantic information expressed by the core content according to the text format requirements of the target generation model, so that the updated text is more compatible with the target generation model, the image data generated for the updated text using the target generation model can better conform to the semantic information expressed by the core content, which is conducive to improving the image generation effect.

[0186] Based on the relevant content of steps 51 to 52 above, it can be known that in some application scenarios, for the above target text, after obtaining the core content of the above target text and the target generation model that matches the core content, the core content can be updated according to the text format constraint information corresponding to the target generation model to obtain an updated text, so that the updated text can express the semantic information expressed by the core content according to the text format requirements of the target generation model, which is conducive to improving the degree of adaptation between the updated text and the target generation model; and then use the target generation model to perform image generation processing on the updated text to obtain image data corresponding to the target text, so that the image data can better conform to the semantic information expressed by the core content, which is conducive to better improving the image generation effect.

[0187] Based on the relevant contents of S1 to S5 above, it can be seen that for the image generation method provided by the embodiment of the present disclosure, after obtaining the target text, a target core content extraction algorithm that matches the target text is first selected from at least two candidate core content extraction algorithms, so that the target core content extraction algorithm can represent the core content extraction algorithm that exists in the at least two candidate core content extraction algorithms and is suitable for performing core content extraction processing on the target text; then, the target core content extraction algorithm is used to perform core content extraction processing on the target text to obtain core content, so that the core content can represent the core semantic information expressed by the target text; then, a target generation model that matches the core content is selected from at least two candidate generation models, so that the target generation model can represent the model that exists in the at least two candidate generation models and is suitable for performing image generation processing on the target text; finally, the target generation model is used to perform image generation processing on the core content to obtain image data corresponding to the target text, so as to achieve the purpose of generating image data from text.

[0188] In addition, since the core content of the above text can represent the core content of the above target text, the defects caused by interference from non-core content in the target text (for example, content with little or no use value, etc.) can be effectively avoided in the related processes implemented based on the core content (for example, the generation model selection process + image generation process), which is conducive to improving the image generation effect.

[0189] In addition, since the core content above can accurately express the core content expressed by the target text (for example, core semantic information, etc.) with as few characters as possible, the related processes implemented based on the core content (for example, the generation model selection process + image generation process) can consume less time, which is conducive to improving the image generation efficiency.

[0190] In addition, since the target core content extraction algorithm mentioned above refers to a core content extraction algorithm selected from at least two candidate core content extraction algorithms mentioned above and is suitable for performing core content extraction processing on the target text, the target core content extraction algorithm has a high degree of adaptability to the target text, so that the core content subsequently extracted from the target text using the target core content extraction algorithm can more accurately represent the core semantic information expressed by the target text, which is conducive to improving the accuracy of core content extraction, thereby improving the image generation effect.

[0191] Furthermore, the present disclosure does not limit the execution subject of the image generation method provided in the embodiments of the present disclosure. For example, the image generation method provided in the embodiments of the present disclosure can be applied to a terminal device or a server. For another example, the image generation method provided in the embodiments of the present disclosure can also be implemented by means of a data interaction process between a terminal device and a server. The terminal device can be a smart phone, a computer, a personal digital assistant (PDA), a tablet computer, etc. The server can be a standalone server, a cluster server, or a cloud server.

[0192] Based on the image generation method provided in the embodiments of the present disclosure, the embodiments of the present disclosure also provide an image generation device, which will be explained and illustrated below in conjunction with Figure 3. Figure 3 is a schematic diagram of the structure of the image generation device provided in the embodiments of the present disclosure. It should be noted that for technical details of the image generation device provided in the embodiments of the present disclosure, please refer to the relevant content of the image generation method above.

[0193] As shown in FIG3 , an image generating apparatus 300 provided in an embodiment of the present disclosure includes:

[0194] A text acquisition unit 301 is used to acquire a target text;

[0195] A first selection unit 302 is configured to select a target core content extraction algorithm that matches the target text from at least two candidate core content extraction algorithms;

[0196] The core content extraction unit 303 is used to perform core content extraction processing on the target text using the target core content extraction algorithm to obtain the core content;

[0197] A second selection unit 304 is configured to select a target generation model that matches the core content from at least two candidate generation models;

[0198] The image generation unit 305 is configured to perform image generation processing on the core content using the target generation model to obtain image data corresponding to the target text.

[0199] In one possible implementation, the first selection unit 302 is specifically used to: determine a target style that matches the target text from the extracted text styles corresponding to the at least two candidate core content extraction algorithms based on the matching degree representation data between the target text and the extracted text styles corresponding to each of the candidate core content extraction algorithms; and determine the candidate core content extraction algorithm corresponding to the target style as the target core content extraction algorithm.

[0200] In one possible implementation, the second selection unit 304 is specifically used to determine the target generation model from the at least two candidate generation models based on the similarity between the core content and the at least two text samples, and the model performance characterization data of the at least two candidate generation models; wherein, for any candidate generation model, the model performance characterization data of the candidate generation model is used to describe the image generation performance presented by the candidate generation model on each of the text samples.

[0201] In a possible implementation manner, the second selecting unit 304 includes:

[0202] A first selection subunit is configured to select at least one similar sample corresponding to the core content from the at least two text samples based on the similarity between the core content and each of the text samples;

[0203] A second selection subunit is configured to select, for any similar sample, at least one performance matching model corresponding to the similar sample from the at least two candidate generation models based on the model performance characterization data of the at least two candidate generation models;

[0204] The first determining subunit is configured to determine the target generation model according to at least one performance matching model corresponding to the at least one similar sample.

[0205] In one possible implementation, the first determination subunit is specifically configured to select the target generation model from the at least one performance matching model corresponding to the at least one similar sample based on model performance characterization data of the at least one performance matching model corresponding to the at least one similar sample.

[0206] In one possible implementation, the first determination subunit is specifically used to: perform statistical analysis on at least one performance matching model corresponding to the at least one similar sample to obtain frequency characterization data of at least one model to be used; the at least one performance matching model corresponding to the at least one similar sample includes the at least one model to be used; for any model to be used, determine the similar sample processing performance data of the model to be used based on the model performance characterization data of the model to be used, so that the similar sample processing performance data of the model to be used is used to describe the image generation performance presented by the model to be used on each of the similar samples; select the target generation model from the at least one model to be used based on the frequency characterization data of the at least one model to be used and the similar sample processing performance data of the at least one model to be used.

[0207] In one possible implementation, the at least two text samples include I text samples, where I is a positive integer; the at least two candidate generation models include J candidate generation models, where J is a positive integer;

[0208] The image generating device 300 further includes:

[0209] A performance determination unit is used to use the jth candidate generation model to perform image generation processing on the i-th text sample to obtain the j-th generated image corresponding to the i-th text sample; j is a positive integer, j≤J, i is a positive integer, i≤I; based on the j-th generated image corresponding to the i-th text sample, determine the i-th sample processing performance characterization data corresponding to the j-th candidate generation model; the i-th sample processing performance characterization data corresponding to the j-th candidate generation model is used to represent the image generation performance presented by the j-th candidate generation model on the i-th text sample; j is a positive integer, j≤J, i is a positive integer, i≤I; based on the 1st sample processing performance characterization data corresponding to the j-th candidate generation model to the 1st sample processing performance characterization data corresponding to the j-th candidate generation model, determine the model performance characterization data of the j-th candidate generation model; j is a positive integer, j≤J.

[0210] In one possible implementation, the i-th sample processing performance characterization data corresponding to the j-th candidate generation model is determined based on the matching degree characterization data between the i-th text sample and the j-th generated image corresponding to the i-th text sample, and the image quality characterization data of the j-th generated image corresponding to the i-th text sample.

[0211] In one possible implementation, the second selection unit 304 is specifically used to: select at least one matching generation model corresponding to the core content from the at least two candidate generation models based on the model description information of the at least two candidate generation models; and determine the target generation model from the at least one matching generation model based on the similarity between the core content and at least two text samples, and the model performance characterization data of the at least one matching generation model.

[0212] In a possible implementation manner, the image generating device 300 further includes:

[0213] A text updating unit, configured to update the core content according to the text format constraint information corresponding to the target generation model to obtain an updated text;

[0214] The image generation unit 305 is specifically configured to perform image generation processing on the updated text using the target generation model to obtain image data corresponding to the target text.

[0215] In a possible implementation manner, the text format constraint information corresponding to the target generation model includes a processed text sample corresponding to the target generation model.

[0216] Based on the relevant content of the above-mentioned image generating device 300, it can be known that for the image generating device 300 provided by the embodiment of the present disclosure, after obtaining the target text, a target core content extraction algorithm that matches the target text is first selected from at least two candidate core content extraction algorithms, so that the target core content extraction algorithm can represent the core content extraction algorithm that exists in the at least two candidate core content extraction algorithms and is suitable for performing core content extraction processing on the target text; then, the target core content extraction algorithm is used to perform core content extraction processing on the target text to obtain the core content, so that the core content can represent the core semantic information expressed by the target text; then, a target generation model that matches the core content is selected from at least two candidate generation models, so that the target generation model can represent the model that exists in the at least two candidate generation models and is suitable for performing image generation processing on the target text; finally, the target generation model is used to perform image generation processing on the core content to obtain image data corresponding to the target text, so as to achieve the purpose of generating image data from text.

[0217] Based on the image generation method provided in the embodiments of the present disclosure, the embodiments of the present disclosure also provide an image generation system, which will be explained and illustrated below in conjunction with Figure 4. Figure 4 is a schematic diagram of the structure of the image generation system provided in the embodiments of the present disclosure. It should be noted that for the technical details of the image generation system provided in the embodiments of the present disclosure, please refer to the relevant content of the image generation method above.

[0218] As shown in FIG4 , an embodiment of the present disclosure provides an image generation system 400, which includes a controller 401 and at least two candidate generation models. The controller 401 is configured to execute any embodiment of the image generation method provided herein. For each candidate generation model, the candidate generation model is configured to perform image generation processing in response to the controller 401. For ease of understanding, the following description is provided in conjunction with some possible implementations.

[0219] In a possible implementation, the controller 401 is configured to obtain a target text;

[0220] The controller 401 is further configured to select a target core content extraction algorithm that matches the target text from at least two candidate core content extraction algorithms;

[0221] The controller 401 is further configured to perform core content extraction processing on the target text using the target core content extraction algorithm to obtain the core content;

[0222] The controller 401 is further configured to select a target generation model that matches the core content from at least two candidate generation models;

[0223] The controller 401 is further configured to call the target generation model to obtain image data corresponding to the target text; the target generation model is configured to perform image generation processing on the core content to obtain image data corresponding to the target text.

[0224] In one possible implementation, the controller 401 is specifically used to: determine a target style that matches the target text from the extracted text styles corresponding to the at least two candidate core content extraction algorithms based on the matching degree representation data between the target text and the extracted text styles corresponding to each of the candidate core content extraction algorithms; and determine the candidate core content extraction algorithm corresponding to the target style as the target core content extraction algorithm.

[0225] In one possible implementation, the controller 401 is specifically used to: determine the target generation model from the at least two candidate generation models based on the similarity between the core content and the at least two text samples, and the model performance characterization data of the at least two candidate generation models; wherein, for any candidate generation model, the model performance characterization data of the candidate generation model is used to describe the image generation performance presented by the candidate generation model on each of the text samples.

[0226] In one possible implementation, the controller 401 is specifically used to: select at least one similar sample corresponding to the core content from the at least two text samples based on the similarity between the core content and each of the text samples; for any similar sample, select at least one performance matching model corresponding to the similar sample from the at least two candidate generation models based on the model performance characterization data of the at least two candidate generation models; and determine the target generation model based on the at least one performance matching model corresponding to the at least one similar sample.

[0227] In one possible implementation, the controller 401 is specifically configured to select the target generation model from the at least one performance matching model corresponding to the at least one similar sample based on model performance characterization data of the at least one performance matching model corresponding to the at least one similar sample.

[0228] In one possible implementation, the controller 401 is specifically used to: perform statistical analysis on at least one performance matching model corresponding to the at least one similar sample to obtain frequency characterization data of at least one model to be used; the at least one performance matching model corresponding to the at least one similar sample includes the at least one model to be used; for any model to be used, determine the similar sample processing performance data of the model to be used based on the model performance characterization data of the model to be used, so that the similar sample processing performance data of the model to be used is used to describe the image generation performance presented by the model to be used on each of the similar samples; select the target generation model from the at least one model to be used based on the frequency characterization data of the at least one model to be used and the similar sample processing performance data of the at least one model to be used.

[0229] In one possible implementation, the at least two text samples include I text samples, where I is a positive integer; the at least two candidate generation models include J candidate generation models, where J is a positive integer;

[0230] The controller 401 is also used to: use the jth candidate generation model to perform image generation processing on the ith text sample to obtain the jth generated image corresponding to the ith text sample; j is a positive integer, j≤J, i is a positive integer, i≤I; determine the i-th sample processing performance characterization data corresponding to the j-th candidate generation model based on the j-th generated image corresponding to the ith text sample; the i-th sample processing performance characterization data corresponding to the j-th candidate generation model is used to represent the image generation performance presented by the j-th candidate generation model on the ith text sample; j is a positive integer, j≤J, i is a positive integer, i≤I; determine the model performance characterization data of the j-th candidate generation model based on the 1st sample processing performance characterization data corresponding to the j-th candidate generation model to the 1st sample processing performance characterization data corresponding to the j-th candidate generation model; j is a positive integer, j≤J.

[0231] In one possible implementation, the i-th sample processing performance characterization data corresponding to the j-th candidate generation model is determined based on the matching degree characterization data between the i-th text sample and the j-th generated image corresponding to the i-th text sample, and the image quality characterization data of the j-th generated image corresponding to the i-th text sample.

[0232] In one possible implementation, the controller 401 is specifically used to: select at least one matching generation model corresponding to the core content from the at least two candidate generation models based on the model description information of the at least two candidate generation models; determine the target generation model from the at least one matching generation model based on the similarity between the core content and at least two text samples, and the model performance characterization data of the at least one matching generation model.

[0233] In one possible implementation, the controller 401 is specifically used to: update the core content based on the text format constraint information corresponding to the target generation model to obtain updated text; and use the target generation model to perform image generation processing on the updated text to obtain image data corresponding to the target text.

[0234] In a possible implementation manner, the text format constraint information corresponding to the target generation model includes a processed text sample corresponding to the target generation model.

[0235] In a possible implementation manner, the controller 401 is a language model.

[0236] Based on the relevant content of the above-mentioned image generation system 400, it can be known that for the image generation system 400 provided by the embodiment of the present disclosure, the working principle of the controller 401 in the image generation system 400 is: after obtaining the target text, first select a target core content extraction algorithm that matches the target text from at least two candidate core content extraction algorithms, so that the target core content extraction algorithm can represent the core content extraction algorithm that exists in the at least two candidate core content extraction algorithms and is suitable for performing core content extraction processing on the target text; then use the target core content extraction algorithm to perform core content extraction processing on the target text to obtain core content, so that the core content can represent the core semantic information expressed by the target text; then, select a target generation model that matches the core content from at least two candidate generation models, so that the target generation model can represent the model that exists in the at least two candidate generation models and is suitable for performing image generation processing on the target text; finally, perform image generation processing on the core content by calling the target generation model to obtain image data corresponding to the target text, so as to achieve the purpose of generating image data from text.

[0237] It can be seen that for the image generation system provided by the present disclosure, the image generation system can generate high-quality images by searching for a generation model that best matches a certain text content, so that the image generation system can have the advantages of high versatility, high practicality and greater convenience. In addition, the image generation system can be composed of a language model and a plurality of candidate generation models, so that the language model in the image generation system can act as a core controller to maintain the entire workflow of the system (for example, core content extraction, sample matching and model performance database matching, execution of image generation and other processes), so that the image generation system can balance the advantages of general image generation models and image generation models in certain specific fields with the help of the language model, and automatically find the best matching model for any text content (for example, text content entered by the user) to perform image generation processing, so that the system can more flexibly achieve the purpose of generating high-quality images from text. In addition, the image generation system can more accurately extract the generation target described by any text content with the help of multiple core content extraction algorithms, which is conducive to improving the image generation effect.

[0238] In addition, an embodiment of the present disclosure also provides an electronic device, which includes a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes any implementation of the image generation method provided by the embodiment of the present disclosure.

[0239] Referring to FIG5 , a schematic diagram of the structure of an electronic device 500 suitable for implementing embodiments of the present disclosure is shown. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG5 is merely an example and should not limit the functionality or scope of use of embodiments of the present disclosure.

[0240] As shown in Figure 5, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0241] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although FIG5 shows the electronic device 500 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0242] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0243] The electronic device provided by the embodiment of the present disclosure and the method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0244] The embodiments of the present disclosure further provide a computer-readable medium, in which instructions or computer programs are stored. When the instructions or computer programs are executed on a device, the device executes any implementation of the image generation method provided by the embodiments of the present disclosure.

[0245] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0246] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0247] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0248] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device can perform the method.

[0249] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0250] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0251] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit / module does not, in some cases, limit the unit itself.

[0252] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0253] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0254] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0255] It should be understood that in the present disclosure, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0256] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0257] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0258] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present disclosure. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments shown herein, but is intended to be construed in the widest manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating an image, the method comprising: Get the target text; Selecting a target core content extraction algorithm that matches the target text from at least two candidate core content extraction algorithms; Using the target core content extraction algorithm to perform core content extraction processing on the target text to obtain the core content; Selecting a target generation model matching the core content from at least two candidate generation models; The target generation model is used to perform image generation processing on the core content to obtain image data corresponding to the target text.

2. The method according to claim 1, wherein: The process of determining the target core content extraction algorithm includes: Determining a target style matching the target text from the extracted text styles corresponding to the at least two candidate core content extraction algorithms according to the matching degree representation data between the target text and the extracted text styles corresponding to the candidate core content extraction algorithms; The candidate core content extraction algorithm corresponding to the target style is determined as the target core content extraction algorithm.

3. The method according to claim 1, wherein: The step of selecting a target generation model matching the core content from at least two candidate generation models comprises: The target generation model is determined from the at least two candidate generation models based on the similarity between the core content and at least two text samples, and the model performance characterization data of the at least two candidate generation models; wherein, for any candidate generation model, the model performance characterization data of the candidate generation model is used to describe the image generation performance presented by the candidate generation model on each of the text samples.

4. The method according to claim 3, wherein: The process of determining the target generation model includes: According to the similarity between the core content and each of the text samples, selecting at least one similar sample corresponding to the core content from the at least two text samples; For any similar sample, selecting at least one performance matching model corresponding to the similar sample from the at least two candidate generation models according to the model performance characterization data of the at least two candidate generation models; The target generation model is determined according to at least one performance matching model corresponding to the at least one similar sample.

5. The method according to claim 4, wherein: The determining the target generation model according to at least one performance matching model corresponding to the at least one similar sample includes: The target generation model is selected from the at least one performance matching model corresponding to the at least one similar sample based on the model performance characterization data of the at least one performance matching model corresponding to the at least one similar sample.

6. The method according to claim 5, wherein: The selecting the target generation model from the at least one performance matching model corresponding to the at least one similar sample according to the model performance characterization data of the at least one performance matching model corresponding to the at least one similar sample comprises: Performing statistical analysis on at least one performance matching model corresponding to the at least one similar sample to obtain occurrence frequency characterization data of at least one model to be used; the at least one performance matching model corresponding to the at least one similar sample includes the at least one model to be used; For any model to be used, the performance of the model to be used is determined according to the model performance characterization data of the model to be used. similar sample processing performance data, so that the similar sample processing performance data of the model to be used is used to describe the image generation performance presented by the model to be used on each of the similar samples; The target generation model is selected from the at least one model to be used based on the occurrence frequency characterization data of the at least one model to be used and the similar sample processing performance data of the at least one model to be used.

7. The method according to claim 3, wherein: The at least two text samples include I text samples, where I is a positive integer; the at least two candidate generation models include J candidate generation models, where J is a positive integer; The process of determining the model performance characterization data of the j-th candidate generation model includes: Using the j-th candidate generation model to perform image generation processing on the i-th text sample, to obtain the j-th generated image corresponding to the i-th text sample; j is a positive integer, j≤J, i is a positive integer, i≤I; According to the jth generated image corresponding to the i-th text sample, determine the i-th sample processing performance characterization data corresponding to the j-th candidate generation model; the i-th sample processing performance characterization data corresponding to the j-th candidate generation model is used to represent the image generation performance presented by the j-th candidate generation model on the i-th text sample; j is a positive integer, j≤J, i is a positive integer, i≤I; The model performance characterization data of the jth candidate generation model is determined based on the 1st sample processing performance characterization data corresponding to the jth candidate generation model to the Ith sample processing performance characterization data corresponding to the jth candidate generation model; j is a positive integer, j≤J.

8. The method according to claim 7, wherein: The i-th sample processing performance characterization data corresponding to the j-th candidate generation model is determined based on the matching degree characterization data between the i-th text sample and the j-th generated image corresponding to the i-th text sample, and the image quality characterization data of the j-th generated image corresponding to the i-th text sample.

9. The method according to claim 3, wherein: Before determining the target generation model from the at least two candidate generation models based on the similarity between the core content and the at least two text samples and the model performance characterization data of the at least two candidate generation models, the method further includes: Selecting at least one matching generation model corresponding to the core content from the at least two candidate generation models according to the model description information of the at least two candidate generation models; The step of determining the target generation model from the at least two candidate generation models based on the similarity between the core content and the at least two text samples and the model performance characterization data of the at least two candidate generation models comprises: The target generation model is determined from the at least one matching generation model according to the similarity between the core content and at least two text samples and the model performance characterization data of the at least one matching generation model.

10. The method according to claim 1, wherein: After selecting a target generation model matching the core content from at least two candidate generation models, the method further includes: According to the text format constraint information corresponding to the target generation model, the core content is updated to obtain an updated text; The using the target generation model to perform image generation processing on the core content to obtain image data corresponding to the target text includes: The target generation model is used to perform image generation processing on the updated text to obtain image data corresponding to the target text.

11. The method according to claim 10, wherein: The text format constraint information corresponding to the target generation model includes processed text samples corresponding to the target generation model.

12. An image generating device, comprising: A text acquisition unit, used for acquiring a target text; A first selection unit, configured to select a target core content extraction algorithm matching the target text from at least two candidate core content extraction algorithms; A core content extraction unit, used to perform core content extraction processing on the target text using the target core content extraction algorithm to obtain the core content; A second selection unit, configured to select a target generation model matching the core content from at least two candidate generation models; An image generation unit is used to perform image generation processing on the core content using the target generation model to obtain image data corresponding to the target text.

13. An image generation system, the image generation system comprising a controller and at least two candidate generation models; The controller is used to obtain the target text; The controller is further configured to select a target core content extraction algorithm that matches the target text from at least two candidate core content extraction algorithms; The controller is further used to perform core content extraction processing on the target text using the target core content extraction algorithm to obtain the core content; The controller is further configured to select a target generation model matching the core content from at least two candidate generation models; The controller is also used to call the target generation model to obtain image data corresponding to the target text; the target generation model is used to perform image generation processing on the core content to obtain image data corresponding to the target text.

14. The system according to claim 13, wherein: The controller is a language model.

15. An electronic device, comprising: Processor and memory; The memory is used to store instructions or computer programs; The processor is used to execute the instructions or computer programs in the memory so that the electronic device executes the method according to any one of claims 1 to 11.

16. A computer-readable medium storing instructions or computer programs, which, when executed on a device, enable the device to execute the method according to any one of claims 1 to 11.

Images