Image generation method, apparatus, and system, electronic device, and computer-readable medium

Abstract

One or more embodiments of the present disclosure disclose an image generation method, apparatus, and system, an electronic device, and a computer-readable medium. The method comprises: after a target text is acquired, first using a pre-constructed model tree to determine, from among at least two candidate generation models, a target generation model corresponding to the target text, so that the target generation model can represent a model which exists among the at least two candidate generation models and which is more suitable for performing image generation processing on the target text; and then using the target generation model to generate image data corresponding to the target text, so that the image data can, as much as possible, conform to semantic information expressed by the target text, and thus the objective of generating image data from text can be achieved.

Description

Image generation method, device, system, electronic device, and computer-readable medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to the Chinese patent application filed with the State Intellectual Property Office of China on September 15, 2023, with application number 202311198565.0 and invention name “Image generation method, device, system, electronic device, computer-readable medium”, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] Embodiments of the present disclosure relate to the field of data processing technology, and in particular, to an image generation method, apparatus, system, electronic device, and computer-readable medium. Background Art

[0004] In some application scenarios (e.g., image generation scenarios, etc.), it may be necessary to perform image generation processing based on the text content provided by the user (e.g., the text content of "a black puppy running") so that the generated image data can represent the semantic information expressed by the text content.

[0005] However, how to realize the requirements described in the previous paragraph is a technical problem that needs to be solved urgently.

[0006] Summary of the Invention

[0007] One or more embodiments of the present disclosure provide an image generation method, apparatus, system, electronic device, and computer-readable medium, which can achieve the purpose of generating image data from text.

[0008] In order to achieve the above objectives, the technical solutions provided by the present disclosure are as follows:

[0009] An embodiment of the present disclosure provides an image generation method, the method comprising:

[0010] Get the target text;

[0011] Determining a target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is used to describe the category attribution status of the at least two candidate generation models;

[0012] The target generation model is used to generate image data corresponding to the target text.

[0013] In one possible implementation, different leaf nodes in the model tree are used to represent different candidate generation models;

[0014] For any leaf node in the model tree, the direct node of the leaf node is used to represent the category to which the candidate generation model represented by the leaf node belongs.

[0015] In one possible implementation, the process of determining the target generation model includes:

[0016] Searching the model tree for non-leaf nodes that match the target text to obtain a matching result;

[0017] According to the matching result, a target generation model corresponding to the target text is determined.

[0018] In one possible implementation, the process of determining the matching result includes:

[0019] Performing core content extraction processing on the target text to obtain core content;

[0020] Search the model tree for non-leaf nodes that match the core content to obtain matching results.

[0021] In one possible implementation, the process of determining the core content includes:

[0022] Determining a target style that matches the target text from the extracted text styles corresponding to at least two core content extraction algorithms;

[0023] The core content extraction algorithm corresponding to the target style is used to perform core content extraction processing on the target text to obtain the core content.

[0024] In a possible implementation manner, before searching the model tree for non-leaf nodes that match the core content and obtaining a matching result, the method further includes:

[0025] Performing key content extraction processing on the core content to obtain at least one key content;

[0026] The step of searching the model tree for non-leaf nodes that match the core content to obtain a matching result includes:

[0027] The model tree is searched for non-leaf nodes that match the at least one key content to obtain a matching result.

[0028] In a possible implementation manner, the model tree includes J non-leaf node aggregation layers arranged in sequence, where J is a positive integer; each non-leaf node aggregation layer includes at least two non-leaf nodes;

[0029] The process of determining the matching result includes:

[0030] Searching for a node matching the target text from the first non-leaf node aggregation layer to obtain a target node corresponding to the first non-leaf node aggregation layer;

[0031] Searching for nodes matching the target text from the search range corresponding to the j-th non-leaf node aggregation layer to obtain the target node corresponding to the j-th non-leaf node aggregation layer; the search range corresponding to the j-th non-leaf node aggregation layer is determined based on the target node corresponding to the j-1-th non-leaf node aggregation layer; j is a positive integer, 2≤j≤J;

[0032] The matching result is determined based on the target nodes corresponding to the J non-leaf node aggregation layers.

[0033] In a possible implementation manner, the parent node of each node in the search range corresponding to the j-th non-leaf node aggregation layer is the target node corresponding to the j-1-th non-leaf node aggregation layer.

[0034] In one possible implementation, determining the target generation model corresponding to the target text based on the matching result includes:

[0035] Determine at least one candidate leaf node corresponding to the matching result from the model tree;

[0036] The target generation model corresponding to the target text is determined based on the data representing the degree of matching between the target text and the model description information corresponding to each candidate leaf node.

[0037] In a possible implementation manner, the model description information includes a model name and at least one model attribute.

[0038] In a possible implementation, the model tree is constructed based on model description information of the at least two candidate generation models, where the model description information includes a model name and at least one model attribute.

[0039] In one possible implementation, the model tree includes at least one non-leaf node aggregation layer and a leaf node aggregation layer; different nodes in the leaf node aggregation layer are used to represent the model names of different candidate generation models; the categories represented by the nodes in the different non-leaf node aggregation layers are determined based on the classification results of the at least two candidate generation models under different attribute dimensions.

[0040] In one possible implementation, the process of determining the image data corresponding to the target text includes:

[0041] updating the target text according to the text format constraint information corresponding to the target generation model to obtain an updated text;

[0042] The target generation model is used to perform image generation processing on the updated text to obtain image data corresponding to the target text.

[0043] In a possible implementation manner, the text format constraint information corresponding to the target generation model includes a processed text sample corresponding to the target generation model.

[0044] An embodiment of the present disclosure provides an image generating device, comprising:

[0045] An acquisition unit, used to acquire target text;

[0046] a determination unit, configured to determine a target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree being configured to describe the category attribution status of the at least two candidate generation models;

[0047] A generating unit is used to generate image data corresponding to the target text using the target generation model.

[0048] An embodiment of the present disclosure provides an image generation system, the image generation system comprising a controller and at least two candidate generation models;

[0049] The controller is used to obtain the target text;

[0050] The controller is further configured to determine a target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is configured to describe the category attribution status of the at least two candidate generation models;

[0051] The controller is further used to call the target generation model to obtain image data corresponding to the target text; the target generation model is used to generate image data corresponding to the target text.

[0052] In one possible implementation, the controller is a language model.

[0053] An embodiment of the present disclosure provides an electronic device, the device comprising: a processor and a memory;

[0054] The memory is used to store instructions or computer programs;

[0055] The processor is configured to execute the instructions or computer program in the memory, so that the electronic device executes the image generation method provided by the embodiment of the present disclosure.

[0056] An embodiment of the present disclosure provides a computer-readable medium, characterized in that the computer-readable medium stores instructions or computer programs, and when the instructions or computer programs are executed on a device, the device executes the image generation method provided by the embodiment of the present disclosure.

[0057] An embodiment of the present disclosure provides a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the image generation method provided by the embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments recorded in the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] FIG1 is a flowchart of an image generation method provided by one embodiment of the present disclosure;

[0060] FIG2 is a schematic diagram of an image generation process provided by an embodiment of the present disclosure;

[0061] FIG3 is a schematic diagram of an idea tree provided by an embodiment of the present disclosure;

[0062] FIG4 is a schematic structural diagram of an image generating device provided by an embodiment of the present disclosure;

[0063] FIG5 is a schematic structural diagram of an image generation system provided by an embodiment of the present disclosure;

[0064] FIG6 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0065] In order to enable those skilled in the art to better understand the solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the embodiments described are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0066] Compared with the related art, one or more embodiments of the present disclosure have at least the following advantages:

[0067] In the technical solution provided by one or more embodiments of the present disclosure, after obtaining the target text, a pre-constructed model tree is first used to determine the target generation model corresponding to the target text from at least two candidate generation models, so that the target generation model can represent the model that exists in the at least two candidate generation models and is more suitable for image generation processing for the target text; then the target generation model is used to generate image data corresponding to the target text, so that the image data can be as consistent as possible with the semantic information expressed by the target text, thereby achieving the purpose of generating image data from text.

[0068] In addition, since the above model tree is used to describe the category affiliation status of at least two candidate generation models above, the model tree can better represent the characteristics of different candidate generation models, so that the target generation model determined subsequently using the model tree for a certain text (for example, the target text above) is more suitable for image generation processing for the text, which can effectively avoid defects caused by the incompatibility between the text and the image generation model (for example, defects such as poor image generation effect), thereby helping to improve the image generation effect.

[0069] To better understand the technical solutions provided by one or more embodiments of the present disclosure, the image generation method provided by one or more embodiments of the present disclosure is described below with reference to some figures. As shown in Figure 1, the image generation method provided by one embodiment of the present disclosure includes S1-S3. Figure 1 is a flowchart of an image generation method provided by one embodiment of the present disclosure.

[0070] S1: Get the target text.

[0071] Among them, the target text refers to the text required as a basis for image generation processing; and the present disclosure does not limit the target text. For example, in some application scenarios, the target text may refer to the text content input by the user with the help of certain input devices (for example, the text content "Generate an image of a dog" shown in Figure 2).

[0072] In addition, the present disclosure does not limit the method for obtaining the target text. For example, in some application scenarios, the target text can be obtained from information input by the user. In another example, in some application scenarios, the target text can be read from some storage space. In another example, in some application scenarios, the target text can refer to text content with image generation processing requirements involved in the execution of other tasks.

[0073] S2: Determine a target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is used to describe the category attribution status of the at least two candidate generation models.

[0074] The candidate generative model refers to a pre-built image generation model with good image generation performance. Therefore, any one of the "at least two candidate generative models" (e.g., Model 1 or Model 2 shown in FIG2 ) can be used to perform image generation processing on the input data of the candidate generative model.

[0075] In addition, in some application scenarios, for the above "at least two candidate generation models", different candidate generation models are suitable for implementing image generation processing in different application scenarios, so that different candidate generation models are suitable for image generation processing for different types of text.

[0076] In addition, the present disclosure does not limit the implementation method of the above candidate generation model. For example, in some application scenarios, the candidate generation model can be implemented using a diffusion model (for example, the diffusion model shown in FIG2 ).

[0077] The model tree is used to express the association relationship (for example, whether they belong to the same category under a certain attribute dimension) between some pre-built image generation models (for example, the "at least two candidate generation models" mentioned above) in the form of a tree structure (for example, a structure such as an idea tree). The image generation model is used to perform image generation processing on the input data of the image generation model. It can be seen that in some application scenarios, the model tree can be used to describe the category attribution status presented by the at least two candidate generation models mentioned above under some attribute dimensions (for example, attribute dimensions such as subject and style).

[0078] Based on the content of the above paragraph, it can be seen that for the above model tree, if the model tree is constructed based on the relevant information of at least two candidate generation models (for example, model attributes and model names, etc.), then the model tree can be used to describe the category attribution status of the at least two candidate generation models. Among them, the category attribution status of the i-th candidate generation model refers to the category attribution status presented by the i-th candidate generation model under at least one attribute dimension, i is a positive integer, i≤I, I is a positive integer, and I represents the number of models in the at least two candidate generation models.

[0079] In addition, for the "at least one attribute dimension" in the above paragraph, the "at least one attribute dimension" is some model classification dimensions obtained by refining the model attributes of the at least two candidate generation models mentioned above; and the present disclosure does not limit the "at least one attribute dimension", for example, it may include dimensions such as subject, style, and language. Among them, the model attributes of the i-th candidate generation model are used to represent the characteristics of the i-th candidate generation model; and the present disclosure does not limit the model attributes of the i-th candidate generation model, for example, it may include at least one model attribute label pre-annotated for the i-th candidate generation model (for example, the label "animals as the subject", the label "ink painting style", the label "Chinese comics", etc.).

[0080] It should be noted that this disclosure does not limit the method for determining the "at least one attribute dimension" in the preceding paragraph. For example, in some application scenarios, the "at least one attribute dimension" can be pre-set based on the actual application scenario. In another example, in some application scenarios, to further enhance flexibility, the "at least one attribute dimension" can be automatically extracted from the model attributes of the at least two candidate generation models.

[0081] In addition, the present disclosure is not limited to the implementation method of the above model tree. For example, it can be implemented with the help of an idea tree (for example, the idea tree shown in FIG. 2 or FIG. 3 ).

[0082] In addition, for the above model tree, the model tree may include at least one non-leaf node aggregation layer (for example, the layers other than the first and last layers in the idea tree shown in FIG2 or FIG3) and a leaf node aggregation layer (for example, the last layer in the idea tree shown in FIG2 or FIG3). The non-leaf node refers to a node with at least one child node in the model tree (for example, nodes 1 to 7 shown in FIG3). The non-leaf node aggregation layer refers to a layer for aggregating and displaying at least two non-leaf nodes (for example, the second or third layer in the idea tree shown in FIG3). The jth non-leaf node aggregation layer refers to the non-leaf node aggregation layer that exists in the model tree and is in the jth arrangement position, so that the jth non-leaf node aggregation layer is used to represent the classification status presented by the at least two candidate generation models above under the jth attribute dimension, so that each node existing in the jth non-leaf node aggregation layer can respectively represent the category to which one or more models of the at least two candidate generation models belong under the jth attribute dimension, j is a positive integer, j is less than or equal to J, J is a positive integer, and J represents the number of non-leaf node aggregation layers in the model tree. The leaf node refers to a node that does not have child nodes in the model tree (for example, nodes 8 to 15 shown in Figure 3). The leaf node aggregation layer refers to a layer used to aggregate and display leaf nodes (for example, the 4th layer in the idea tree shown in Figure 3), and different nodes in the leaf node aggregation layer are used to represent the model names of different candidate generation models (for example, node 8 shown in Figure 3 is used to represent the model name "Model 3", node 9 shown in Figure 3 is used to represent the model name "Model 4", etc.), so that different nodes in the leaf node aggregation layer can be used to represent different candidate generation models, so that the leaf node aggregation layer can be used to represent the at least two candidate generation models, and then the model tree can describe the category attribution status of the at least two candidate generation models.

[0083] Based on the content of the above paragraph, it can be known that in some application scenarios, for the above model tree (for example, the idea tree shown in Figure 3), the model tree may include K layers (for example, 4 layers as shown in Figure 3), where K is a positive integer. Among them, the first layer of the model tree is used to record the root node (for example, node 1 shown in Figure 3). The kth layer of the model tree is used to record the classification status presented by the above at least two candidate generation models under the k-1th attribute dimension, so that the kth layer can represent the k-1th non-leaf node aggregation layer existing in the model tree, so that the category name represented by each node in the kth layer can represent the category to which one or more models in the above at least two candidate generation models belong under the k-1th attribute dimension, k is a positive integer, 2≤k≤K-1. The Kth layer of the model tree is used to record the leaf nodes corresponding to the at least two candidate generation models, so that the model names represented by each leaf node existing in the Kth layer respectively identify each candidate generation model, so that different nodes in the Kth layer respectively represent different candidate generation models.

[0084] Based on the content of the above paragraph, it can be known that under a possible implementation method, for the above model tree, different leaf nodes in the model tree are used to represent different candidate generation models; and for any leaf node in the model tree, the direct node of the leaf node is used to represent the category to which the candidate generation model represented by the leaf node belongs. Among them, the direct node of the qth leaf node refers to a node existing in the model tree that has a certain direct relationship with the qth leaf node (for example, a father-child relationship, a grandparent-grandchild relationship, etc.), so that the direct node of the qth leaf node can represent the non-leaf node that exists in the model tree and appears in the shortest path with the qth leaf node as the starting point and the root node of the model tree as the end point. q is a positive integer, q≤Q, Q is a positive integer, and Q represents the number of leaf nodes in the model tree.

[0085] In addition, the present disclosure does not limit the implementation of the direct nodes of the qth leaf node above. For example, in some application scenarios, the direct nodes of the qth leaf node may include all non-leaf nodes in the shortest path corresponding to the qth leaf node. The shortest path corresponding to the qth leaf node refers to the shortest path existing in the above model tree, with the qth leaf node as the starting point and the root node of the model tree as the end point.

[0086] It can be seen that in one possible implementation, for the qth leaf node in the above model tree (for example, node 12 shown in Figure 3), the shortest path corresponding to the qth leaf node can be the path of the qth leaf node → the parent node of the qth leaf node → the grandparent node of the qth leaf node → ... (for example, the path of node 12 → node 6 → node 3 → node 1 shown in Figure 3); and the direct nodes of the qth leaf node may specifically include the parent node of the qth leaf node (for example, node 6 as the parent node of node 12 shown in Figure 3), the grandparent node of the qth leaf node (that is, the parent node of the parent node of the qth leaf node, for example, node 3 as the grandparent node of node 12 shown in Figure 3), the great-grandparent node of the qth leaf node (that is, the parent node of the grandparent node of the qth leaf node, for example, node 1 as the great-grandparent node of node 12 shown in Figure 3), ... (and so on).

[0087] For example, in some application scenarios, the direct descendants of the qth leaf node described above may include all non-leaf nodes in the shortest path corresponding to the qth leaf node, excluding the root node. For example, when the qth leaf node is node 12 shown in Figure 3 , the shortest path corresponding to the qth leaf node may be node 12 → node 6 → node 3 → node 1; and the direct descendants of the qth leaf node may specifically include node 6 and node 3.

[0088] Based on the relevant content of the direct nodes of the qth leaf node above, it can be known that for the qth leaf node, the direct nodes of the qth leaf node can include a node in the shortest path corresponding to the qth leaf node in each non-leaf node aggregation layer in the above model tree, so that the different direct nodes of the qth leaf node can represent the category to which the candidate generation model represented by the qth leaf node belongs under different attribute dimensions, thereby enabling all the direct nodes of the qth leaf node to represent the category affiliation status of the candidate generation model represented by the qth leaf node under at least one attribute dimension. It can be seen that when the model tree includes J non-leaf node aggregation layers, the direct nodes of the qth leaf node can include the direct nodes corresponding to the qth leaf node in the first non-leaf node aggregation layer, the direct nodes corresponding to the qth leaf node in the second non-leaf node aggregation layer, ... (and so on), and the direct nodes corresponding to the qth leaf node in the Jth non-leaf node aggregation layer. Among them, the direct node corresponding to the qth leaf node in the jth non-leaf node aggregation layer refers to the node passed by the shortest path corresponding to the qth leaf node in the jth non-leaf node aggregation layer, so that the "direct node corresponding to the qth leaf node in the jth non-leaf node aggregation layer" can represent the category of the candidate generation model represented by the qth leaf node under the jth attribute dimension, j is a positive integer, j≤J, and J is a positive integer.

[0089] Based on the relevant content of the model tree above, it can be known that in some application scenarios, for the model tree, the leaf nodes in the model tree can be used to represent a candidate generation model, and the non-leaf nodes in the model tree other than the root node can be used to represent the category to which a candidate generation model belongs under a certain attribute dimension. Based on this, it can be known that in a possible implementation, in order to meet the above requirements, for any leaf node in the model tree, the model identifier of a candidate generation model (for example, identification information such as the model name) can be used as the node name of the leaf node, so that the leaf node can be used to represent the model identifier of the candidate generation model, so that the leaf node can represent the candidate generation model; and, for any non-leaf node in the model tree other than the root node, the category identifier (for example, the category name) of the category to which a candidate generation model belongs under the corresponding attribute dimension can be used as the node name of the non-leaf node, so that the non-leaf node is used to represent the category identifier of the category to which the candidate generation model belongs under the corresponding attribute dimension, so that the non-leaf node can represent the category to which the candidate generation model belongs under the corresponding attribute dimension.

[0090] Based on the above content, it can be seen that in one possible implementation, the above model tree can include at least one non-leaf node aggregation layer (for example, the second and third layers of the idea tree shown in Figure 3) and a leaf node aggregation layer (for example, the fourth layer of the idea tree shown in Figure 3). Different nodes in the leaf node aggregation layer are used to represent the model names of different candidate generation models; and nodes in different non-leaf node aggregation layers are used to represent categories under different attribute dimensions.

[0091] In addition, the present disclosure does not limit the construction process of the above model tree. For example, in some application scenarios, the model tree can be manually constructed by relevant personnel.

[0092] For another example, in some application scenarios, the above model tree can be automatically constructed based on the relevant information (such as model name, model attributes, etc.) of the above at least two candidate generation models. It can be seen that, in one possible implementation, the model tree can be constructed based on the model description information of the above at least two candidate generation models. Among them, the model description information of the i-th candidate generation model is used to describe the characteristics of the i-th candidate generation model; and the present disclosure does not limit the implementation method of the model description information of the i-th candidate generation model. For example, in one possible implementation, the model description information of the i-th candidate generation model may include the model name of the i-th candidate generation model and at least one model attribute of the i-th candidate generation model. Among them, the model name of the i-th candidate generation model is used to uniquely identify the i-th candidate generation model; and the present disclosure does not limit the model name of the i-th candidate generation model. The at least one model attribute of the i-th candidate generation model is used to represent the attribute state of the candidate generation model; and the present disclosure does not limit the at least one model attribute of the i-th candidate generation model. i is a positive integer, i≤I, I is a positive integer, and I represents the number of models in the at least two candidate generation models.

[0093] In addition, the present disclosure does not limit the automatic construction process of the model tree in the above paragraph. For example, in one possible implementation, the construction process of the model tree may specifically include the following steps 11 and 12.

[0094] Step 11: Obtain at least one attribute dimension.

[0095] It should be noted that, for the relevant content of step 11, please refer to the relevant content of "at least one attribute dimension" above. For the sake of brevity, it will not be repeated here.

[0096] Step 12: Based on the model names of the at least two candidate generation models above and the classification results of the at least two candidate generation models under each attribute dimension, construct a model tree so that different nodes in the leaf node aggregation layer of the model tree are used to represent the model names of different candidate generation models, and nodes in different non-leaf node aggregation layers of the model tree are used to represent categories under different attribute dimensions.

[0097] In the present disclosure, after obtaining the classification results of the above at least two candidate generation models under the j-th attribute dimension, the j-th non-leaf node aggregation layer in the above model tree can be constructed based on the classification results, so that different nodes in the j-th non-leaf node aggregation layer can represent different categories under the j-th attribute dimension, so that the j-th non-leaf node aggregation layer can represent the classification status of the at least two candidate generation models under the j-th attribute dimension, j is a positive integer, j≤J, J is a positive integer, so that after all non-leaf node aggregation layers in the model tree are constructed, the nodes used to represent the model names of each candidate generation model are added to the model tree as leaf nodes, so that different nodes in the leaf node aggregation layer of the finally constructed model tree are used to represent the model names of different candidate generation models, and the nodes in different non-leaf node aggregation layers of the model tree are used to represent categories under different attribute dimensions, so that the finally constructed model tree can represent the category attribution status of the at least two candidate generation models.

[0098] Based on the relevant content of steps 11 to 12 above, it can be known that in some application scenarios, when the above model tree can include at least one non-leaf node aggregation layer (for example, the 2nd and 3rd layers of the idea tree shown in Figure 3), the categories represented by the nodes in different non-leaf node aggregation layers are determined based on the classification results of the above at least two candidate generation models under different attribute dimensions, so that the model tree is used to describe the category affiliation status of the at least two candidate generation models (for example, information such as the category under different attribute dimensions).

[0099] Based on the relevant content of the model tree above, it can be known that for the model tree provided by the present disclosure, the model tree can be used to describe the category affiliation status of at least two candidate generation models above, so that the model tree can represent the classification results of these candidate generation models under different attribute dimensions, thereby enabling the model tree to better represent the characteristics of these candidate generation models, which is conducive to improving the execution efficiency of selecting a model suitable for image generation processing for the target text from these candidate generation models.

[0100] The "target generation model corresponding to the target text" mentioned above refers to a model selected from the at least two candidate generation models mentioned above using the model tree, and is suitable for performing image generation processing for the target text, so that the target generation model can be used to perform image generation processing for the target text. For example, when the model tree is the idea tree shown in Figure 2, the "target generation model corresponding to the target text" may refer to Model 1.

[0101] In addition, the present disclosure does not limit the determination process of the above “target generation model corresponding to the target text”. For ease of understanding, some possible implementation methods are described below.

[0102] In a possible implementation, the process of determining the “target generation model corresponding to the target text” mentioned above may specifically include the following steps 21 and 22.

[0103] Step 21: Search for non-leaf nodes that match the target text from the model tree to obtain a matching result, so that the matching result can represent the category affiliation status of the image generation model suitable for performing image generation processing on the target text.

[0104] The matching result is used to describe the non-leaf nodes in the above model tree that match the target text, so that the matching result can represent the category belonging status of the image generation model that is suitable for performing image generation processing on the target text. For example, when the model tree is the idea tree shown in Figure 3, the matching result can include node 3 and node 6, so that the matching result can represent that the image generation model that is suitable for performing image generation processing on the target text belongs to the category represented by node 3 (for example, the subject category of "animal") and the category represented by node 6 (for example, the style category of "ink painting").

[0105] In addition, the present disclosure does not limit the process of determining the above matching results. For example, it can be implemented using any existing or future method that can search and process a tree structure (such as a breadth-first search method, etc.).

[0106] In fact, in some application scenarios, there may be some content with less use value in the above target text. Therefore, in order to improve the matching efficiency, the present disclosure also provides a possible implementation method of the process of determining the above matching results. Under this implementation method, the process of determining the matching results can specifically include the following steps 211-212.

[0107] Step 211: Perform core content extraction processing on the target text to obtain the core content.

[0108] The core content is used to represent the core content described in the target text. For example, when the target text is the text "Generate an image of a dog" as shown in FIG2, the core content may be the content "image of a dog".

[0109] In addition, the present disclosure does not limit the implementation of step 211. For example, it can be implemented by any existing or future method that can extract the core content of a text content.

[0110] In addition, in order to better improve the accuracy of core content extraction, the present disclosure also provides a possible implementation of the above step 211. Under this implementation, the step 211 may specifically include the following steps 2111-2112.

[0111] Step 2111: Determine a target style that matches the above target text from the extracted text styles corresponding to at least two core content extraction algorithms.

[0112] The core content extraction algorithm is used to extract the core content of a text.

[0113] In addition, for at least two core content extraction algorithms above (for example, extraction algorithm 1 to extraction algorithm N shown in Figure 2, N is a positive integer), different core content extraction algorithms have different working principles. For example, in some application scenarios, the at least two core content extraction algorithms may include a prompt core content extraction algorithm, an instruction core content extraction algorithm, an inspiration core content extraction algorithm, and a hypothesis core content extraction algorithm. Among them, the prompt core content extraction algorithm is more suitable for performing core content extraction processing on the text content expressed in accordance with the prompt style (for example, the text content of "a dog"), and the working principle of the prompt core content extraction algorithm can be specifically as follows: directly treating the entire text as the core content (for example, the core content of "a dog"). The instruction core content extraction algorithm is more suitable for performing core content extraction processing on the text content expressed in accordance with the instruction style (for example, the text content of "generating an image of a dog"), and the working principle of the instruction core content extraction algorithm can be specifically as follows: extracting the core part behind the instruction as the core content (for example, the core content of "an image of a dog"). The inspiration core content extraction algorithm is more suitable for extracting core content from text content expressed in an inspirational style (for example, the text content "I want to go to the beach"). The working principle of the inspiration core content extraction algorithm can be specifically as follows: extracting the target subject of desire as the core content (for example, the core content "beach"). The hypothesis core content extraction algorithm is more suitable for extracting core content from text content expressed in a hypothesis style (for example, the text content "If you give me a bouquet of flowers, I will smile happily"), and the working principle of the hypothesis core content extraction algorithm can be specifically as follows: extracting the hypothesis condition and the object of the action to be collected as the core content (for example, the core content "a bouquet of flowers and a smiling face").

[0114] For the at least two core content extraction algorithms mentioned above (for example, extraction algorithm 1 to extraction algorithm N shown in Figure 2, where N is a positive integer), different core content extraction algorithms are suitable for performing core content extraction processing on texts of different styles, so that the at least two core content extraction algorithms can be applicable to performing core content extraction processing on texts of multiple styles (for example, text content expressed in a prompt style, text content expressed in an instruction style, text content expressed in an inspiration style, or text content expressed in a hypothesis style, etc.), which is conducive to improving the accuracy of core content extraction.

[0115] In addition, for the nth core content extraction algorithm, if the nth core content extraction algorithm is applicable to the core content extraction processing of the text content of the nth style, then the extracted text style corresponding to the nth core content extraction algorithm is the nth style (for example, prompt style, instruction style, inspiration style, or hypothesis style, etc.). It should be noted that the present disclosure does not limit the implementation method of the above nth style. For example, it can be set in advance according to the actual application scenario. Wherein, n is a positive integer, n≤N, N is a positive integer, and N represents the number of algorithms in the at least two core content extraction algorithms above.

[0116] The target style refers to a style that exists in the extracted text styles corresponding to the at least two core content extraction algorithms above and matches the target text, so that the target style can represent the text style that the target text conforms to.

[0117] In addition, the present disclosure does not limit the determination process of the target style mentioned above. For example, it can be specifically as follows: after obtaining the target text, determine whether the target text conforms to the extraction text style corresponding to the nth core content extraction algorithm. If so, determine the extraction text style corresponding to the nth core content extraction algorithm as the target style, so that the target text can be subsequently processed by using the nth core content extraction algorithm corresponding to the target style. Wherein, n is a positive integer, n≤N, N is a positive integer, and N represents the number of algorithms in the at least two core content extraction algorithms mentioned above.

[0118] Based on the relevant content of step 2111 above, it can be known that after obtaining the target text, a target style that matches the above target text can be determined from the extracted text styles corresponding to some pre-specified core content extraction algorithms with better core content extraction performance, so that the core content extraction algorithm corresponding to the target style can be used to perform core content extraction processing on the target text.

[0119] Step 2112: Using the core content extraction algorithm corresponding to the target style above, perform core content extraction processing on the target text above to obtain the core content.

[0120] The core content extraction algorithm corresponding to the target style is used to represent a method for extracting core content from text content expressed in accordance with the target style. For example, if the target style is a prompt style, the core content extraction algorithm corresponding to the target style may be a prompt core content extraction algorithm; if the target style is an instruction style, the core content extraction algorithm corresponding to the target style may be an instruction core content extraction algorithm; if the target style is an inspiration style, the core content extraction algorithm corresponding to the target style may be an inspiration core content extraction algorithm; if the target style is a hypothesis style, the core content extraction algorithm corresponding to the target style may be a hypothesis core content extraction algorithm.

[0121] Based on the relevant content of steps 2111 to 2112 above, it can be known that in some application scenarios, after obtaining the text content input by the user (for example, the target text above), it can be determined first which core content extraction algorithm among the at least two core content extraction algorithms above has the text style closer to the text style corresponding to the extraction text, so that after determining that the text style presented by the text content is closest to the extraction text style corresponding to the target core content extraction algorithm, the target core content extraction algorithm is directly used to perform core content extraction processing on the text content to obtain the core content, so that the core content can express the core information carried by the text content, which is conducive to eliminating the impact of noise text on the image generation process, thereby facilitating the generation of high-quality images. Among them, the target core content extraction algorithm refers to the algorithm that is most suitable for performing core content extraction processing on the text content among the at least two core content extraction algorithms.

[0122] Based on the relevant content of step 211 above, it can be known that in some application scenarios, after obtaining the target text input by the user, the target text can be subjected to core content extraction processing to obtain the core content, so that the core content can represent the core information carried by the target text. This is conducive to eliminating the impact of noise text on the image generation process, thereby facilitating the generation of high-quality images.

[0123] Step 212: Search the model tree for non-leaf nodes that match the core content above to obtain matching results.

[0124] It should be noted that the present disclosure does not limit the implementation of the above step 212. For example, it can be specifically: according to a preset search method, search for non-leaf nodes that match the above core content from the model tree to obtain a matching result, so that the matching result can indicate which non-leaf nodes in the model tree the core content matches, so that the matching result can indicate the category of the model suitable for image processing of the above target text. The preset search algorithm is used to search for certain content from the model tree; and the present disclosure does not limit the implementation of the preset search algorithm. For example, it can be implemented using a breadth-first search algorithm.

[0125] Based on the content of the above paragraph, it can be seen that in a possible implementation method, the above step 212 can be specifically: searching for nodes that match the above core content from each non-leaf node aggregation layer of the model tree in turn to obtain matching results, so that the matching results can represent the nodes that match the core content obtained by searching from each non-leaf node aggregation layer.

[0126] It should be noted that the present disclosure does not limit the implementation method of the step of "searching for nodes matching the above core content from each non-leaf node aggregation layer of the model tree in turn" in the above paragraph. For example, it can be specifically: searching for nodes matching the above core content from the first non-leaf node aggregation layer of the model tree, and obtaining the matching nodes corresponding to the first non-leaf node aggregation layer, so that the matching nodes corresponding to the first non-leaf node aggregation layer can represent the category that best matches the core content under the first attribute dimension; searching for nodes matching the core content from all child nodes of the matching node corresponding to the first non-leaf node aggregation layer (that is, the nodes in the second non-leaf node aggregation layer that have the matching node corresponding to the first non-leaf node aggregation layer as the parent node), and obtaining the matching nodes corresponding to the second non-leaf node aggregation layer, so that the matching nodes corresponding to the second non-leaf node aggregation layer can represent the category that best matches the core content under the second attribute dimension. Differentiation; searching for nodes that match the core content from all child nodes of the matching node corresponding to the second non-leaf node aggregation layer (that is, the nodes in the third non-leaf node aggregation layer that have the matching node corresponding to the second non-leaf node aggregation layer as a parent node), and obtaining the matching nodes corresponding to the third non-leaf node aggregation layer, so that the matching nodes corresponding to the third non-leaf node aggregation layer can represent the category that best matches the core content under the third attribute dimension; ... (and so on), so that the matching result can be determined subsequently based on the matching nodes corresponding to all non-leaf node aggregation layers (for example, the matching nodes corresponding to the first non-leaf node aggregation layer, the matching nodes corresponding to the second non-leaf node aggregation layer, ...), so that the matching result includes the matching nodes corresponding to all non-leaf node aggregation layers, so that the matching result can represent the category attribution status of the model suitable for image processing of the target text above under each attribute dimension.

[0127] Based on the relevant content of steps 211 to 212 above, it can be seen that in some application scenarios, after obtaining the target text input by the user, the target text can be first subjected to core content extraction processing to obtain the core content, so that the core content can represent the core information carried by the target text; then search the model tree for non-leaf nodes that match the core content to obtain a matching result, so that the matching result can better represent the category affiliation status of the image generation model suitable for image generation processing for the target text.

[0128] In fact, in some application scenarios, in order to better improve the matching efficiency, the present disclosure also provides another possible implementation of the above step 21. In this implementation, the step 21 may specifically include the following steps 213 to 215.

[0129] Step 213: Perform core content extraction processing on the target text to obtain the core content.

[0130] It should be noted that for the relevant content of step 213 above, please refer to the relevant content of step 211 above. For the sake of brevity, it will not be repeated here.

[0131] Step 214: Perform key content extraction processing on the core content of the above text to obtain at least one key content.

[0132] Among them, the key content extraction processing is used to extract some content with higher value (such as keywords, key phrases, etc.) from a text content; and the present disclosure does not limit the implementation method of the key content extraction processing. For example, it can be implemented using any existing or future method that can extract content with higher value from a text content (such as keyword extraction method, etc.).

[0133] It should be noted that the present disclosure does not limit the implementation method of the key content. For example, it can be implemented using keywords, key words, key sentences, etc.

[0134] In addition, for the at least one key content above, the at least one key content is used to represent information with higher value present in the core content above; and the present disclosure does not limit the implementation method of the at least one key content, for example, it may include one or more of at least one keyword, at least one keyword and at least one key sentence present in the core content.

[0135] Step 215: Search the model tree for a non-leaf node that matches at least one key content in the above text to obtain a matching result.

[0136] It should be noted that the present disclosure does not limit the implementation of the above step 215. For example, it can be specifically: according to a preset search method, search for non-leaf nodes that match at least one key content in the model tree to obtain a matching result, so that the matching result can indicate which non-leaf nodes in the model tree the at least one key content matches, so that the matching result can indicate the category status of the model suitable for image processing of the target text in the above. The preset search algorithm is used to search for certain content from the model tree; and the present disclosure does not limit the implementation of the preset search algorithm. For example, it can be implemented using a breadth-first search algorithm.

[0137] Based on the content of the above paragraph, it can be seen that in a possible implementation method, the above step 215 can be specifically as follows: searching for nodes that match at least one key content above from each non-leaf node aggregation layer of the model tree in turn, and obtaining a matching result, so that the matching result can represent the nodes that match the at least one key content obtained by searching from each non-leaf node aggregation layer.

[0138] It should be noted that the present disclosure does not limit the implementation method of the step of "searching for nodes that match at least one key content above from each non-leaf node aggregation layer of the model tree in turn" in the above paragraph. For example, it can be specifically: searching for nodes that match at least one key content above from the first non-leaf node aggregation layer of the model tree, and obtaining matching nodes corresponding to the first non-leaf node aggregation layer, so that the matching nodes corresponding to the first non-leaf node aggregation layer can represent the category that best matches the at least one key content under the first attribute dimension; searching for nodes that match the at least one key content from all child nodes of the matching node corresponding to the first non-leaf node aggregation layer (that is, the nodes that exist in the second non-leaf node aggregation layer with the first non-leaf node as the key content) Searching for nodes that match the at least one key content from all child nodes of the matching node corresponding to the second non-leaf node aggregation layer (that is, nodes in the third non-leaf node aggregation layer with the matching node corresponding to the second non-leaf node aggregation layer as a parent node) to obtain the matching node corresponding to the second non-leaf node aggregation layer, so that the matching node corresponding to the second non-leaf node aggregation layer can represent the category that best matches the at least one key content under the second attribute dimension; searching for nodes that match the at least one key content from all child nodes of the matching node corresponding to the second non-leaf node aggregation layer (that is, nodes in the third non-leaf node aggregation layer with the matching node corresponding to the second non-leaf node aggregation layer as a parent node) to obtain the matching node corresponding to the third non-leaf node aggregation layer, so that the matching node corresponding to the third non-leaf node aggregation layer can represent the category that best matches the at least one key content under the second attribute dimension. The matching nodes corresponding to the first and second non-leaf node aggregation layers can represent the category that best matches the at least one key content under the third attribute dimension; ... (and so on), so that the matching results can be determined subsequently based on the matching nodes corresponding to all non-leaf node aggregation layers (for example, the matching nodes corresponding to the first non-leaf node aggregation layer, the matching nodes corresponding to the second non-leaf node aggregation layer, ...), so that the matching results include the matching nodes corresponding to all non-leaf node aggregation layers, so that the matching results can represent the category affiliation status of the model suitable for image processing of the target text above under each attribute dimension.

[0139] Based on the relevant content of steps 213 to 215 above, it can be seen that in some application scenarios, after obtaining the target text input by the user, the target text can be first subjected to core content extraction processing to obtain the core content, so that the core content can represent the core information carried by the target text; then at least one keyword content is extracted from the core content, so that the at least one keyword content can represent the core information carried by the target text as briefly as possible; then a non-leaf node matching the at least one keyword content is searched from the model tree to obtain a matching result, so that the matching result can better represent the category affiliation status of the image generation model suitable for image generation processing for the target text.

[0140] In fact, in some application scenarios, in order to better improve the matching efficiency, the present disclosure also provides another possible implementation of the above step 21. In this implementation, when the above model tree includes J non-leaf node aggregation layers arranged in sequence (for example, the 2nd to 3rd layers in the idea tree shown in Figure 3), each non-leaf node aggregation layer includes at least two non-leaf nodes, and J is a positive integer, the step 21 can specifically include the following steps 216-218.

[0141] Step 216: Search the first non-leaf node aggregation layer for a node that matches the target text, and obtain the target node corresponding to the first non-leaf node aggregation layer.

[0142] The first non-leaf node aggregation layer refers to the non-leaf node aggregation layer that is arranged at the front in the model tree. For example, when the model tree is the idea tree shown in FIG3 , the first non-leaf node aggregation layer can be the second layer in the idea tree shown in FIG3 .

[0143] In addition, for the first non-leaf node aggregation layer in the above model tree (for example, the second layer in the idea tree shown in Figure 3), the first non-leaf node aggregation layer includes at least two non-leaf nodes (for example, node 2 and node 3 shown in Figure 3); and the parent node of each node in the first non-leaf node aggregation layer is the root node of the model tree (for example, node 1 shown in Figure 3).

[0144] In addition, for the first non-leaf node aggregation layer in the above model tree, the target node corresponding to the first non-leaf node aggregation layer refers to the node in the first non-leaf node aggregation layer that matches the semantic information carried by the target text, so that the target node corresponding to the first non-leaf node aggregation layer can represent the category that matches the target text under the first attribute dimension. For example, when the first non-leaf node aggregation layer is the second layer in the idea tree shown in Figure 3, the target node corresponding to the first non-leaf node aggregation layer can refer to node 3 shown in Figure 3.

[0145] In addition, the present disclosure does not limit the implementation method of the above step 216. For example, in some application scenarios, the step 216 can specifically be: after obtaining the target text, directly search for nodes matching the target text from the first non-leaf node aggregation layer of the model tree to obtain the target node corresponding to the first non-leaf node aggregation layer.

[0146] For example, in some application scenarios, the above step 216 can be specifically as follows: after obtaining the target text, first perform core content extraction processing on the target text to obtain the core content; then search for nodes matching the core content from the first non-leaf node aggregation layer of the model tree to obtain the target node corresponding to the first non-leaf node aggregation layer.

[0147] For example, in some application scenarios, the above step 216 can be specifically as follows: after obtaining the target text, first perform core content extraction processing on the target text to obtain the core content; then perform key content extraction processing on the core content to obtain at least one key content; then, search for nodes corresponding to the at least one key content from the first non-leaf node aggregation layer of the model tree to obtain the target node corresponding to the first non-leaf node aggregation layer.

[0148] Based on the relevant content of step 216 above, it can be known that in some application scenarios, after obtaining the target text, the node matching the target text can be searched from the first non-leaf node aggregation layer of the model tree to obtain the target node corresponding to the first non-leaf node aggregation layer (for example, node 3 shown in Figure 3), so that the matching result corresponding to the target text can be determined based on the target node in the future.

[0149] Step 217: Search for nodes that match the target text from the search range corresponding to the j-th non-leaf node aggregation layer to obtain the target node corresponding to the j-th non-leaf node aggregation layer; the search range corresponding to the j-th non-leaf node aggregation layer is determined based on the target node corresponding to the j-1-th non-leaf node aggregation layer; j is a positive integer, 2≤j≤J.

[0150] The j-th non-leaf node aggregation layer refers to the non-leaf node aggregation layer at the j-th arrangement position in the above model tree, where j is a positive integer, 2≤j≤J.

[0151] In addition, for the j-th non-leaf node aggregation layer in the above model tree (for example, the third layer in the idea tree shown in FIG3 ), the j-th non-leaf node aggregation layer includes at least two non-leaf nodes (for example, nodes 4 to 7 shown in FIG3 ); and the parent node of each node in the j-th non-leaf node aggregation layer is located in the j-1-th non-leaf node aggregation layer in the model tree (for example, the second layer in the idea tree shown in FIG3 ). The j-1-th non-leaf node aggregation layer refers to the non-leaf node aggregation layer in the j-1-th arrangement position in the above model tree, j is a positive integer, and 2≤j≤J.

[0152] Based on the content in the previous paragraph, it can be seen that for the above model tree, the parent node of each node in the second non-leaf node aggregation layer in the model tree is located in the first non-leaf node aggregation layer; the parent node of each node in the third non-leaf node aggregation layer in the model tree is located in the second non-leaf node aggregation layer; ... (and so on); the parent node of each node in the Jth non-leaf node aggregation layer in the model tree is located in the J-1th non-leaf node aggregation layer.

[0153] In addition, for the j-th non-leaf node aggregation layer in the above model tree, the search range corresponding to the j-th non-leaf node aggregation layer is determined based on the target node corresponding to the j-1-th non-leaf node aggregation layer, so that the parent node of each node in the search range corresponding to the j-th non-leaf node aggregation layer is the target node corresponding to the j-1-th non-leaf node aggregation layer. The search range corresponding to the j-th non-leaf node aggregation layer refers to the search range required to be based on when searching for the j-th non-leaf node aggregation layer when the search result for the j-1-th non-leaf node aggregation layer is determined; and the search range corresponding to the j-th non-leaf node aggregation layer may include at least one node existing in the j-th non-leaf node aggregation layer, which has the target node corresponding to the j-1-th non-leaf node aggregation layer as a parent node, j is a positive integer, 2≤j≤J.

[0154] Based on the content of the above paragraph, it can be known that in one possible implementation, the search range corresponding to the j-th non-leaf node aggregation layer above may include at least one child node of the target node corresponding to the j-1-th non-leaf node aggregation layer. For example, when the j-th non-leaf node aggregation layer is the third layer in the idea tree shown in FIG3 , and the target node corresponding to the j-1-th non-leaf node aggregation layer is node 3 shown in FIG3 , the search range corresponding to the j-th non-leaf node aggregation layer may include nodes 6 and 7 shown in FIG3 . Wherein, j is a positive integer, 2≤j≤J.

[0155] In addition, for the j-th non-leaf node aggregation layer in the above model tree, the target node corresponding to the j-th non-leaf node aggregation layer refers to the node that matches the semantic information carried by the target text and exists in the search range corresponding to the j-th non-leaf node aggregation layer, so that the target node corresponding to the j-th non-leaf node aggregation layer can represent the category that matches the target text under the j-th attribute dimension, j is a positive integer, 2≤j≤J. For example, when the j-th non-leaf node aggregation layer is the second layer in the idea tree shown in Figure 3, and the search range corresponding to the j-th non-leaf node aggregation layer includes nodes 6 and 7 shown in Figure 3, the target node corresponding to the j-th non-leaf node aggregation layer may refer to node 6 shown in Figure 3.

[0156] In addition, the present disclosure does not limit the implementation method of the above step 217. For example, it can be specifically as follows: after determining the target node corresponding to the j-1th non-leaf node aggregation layer in the above model tree, first determine the search range corresponding to the j-1th non-leaf node aggregation layer based on the target node corresponding to the j-1th non-leaf node aggregation layer, so that the search range includes at least one child node of the target node corresponding to the j-1th non-leaf node aggregation layer; then directly search for nodes matching the target text from the search range corresponding to the j-1th non-leaf node aggregation layer to obtain the target node corresponding to the j-1th non-leaf node aggregation layer. Wherein, j is a positive integer, 2≤j≤J.

[0157] For another example, in some application scenarios, the above step 217 can be specifically as follows: after determining the target node corresponding to the j-1th non-leaf node aggregation layer in the above model tree, first determine the search range corresponding to the j-1th non-leaf node aggregation layer based on the target node corresponding to the j-1th non-leaf node aggregation layer, so that the search range includes at least one child node of the target node corresponding to the j-1th non-leaf node aggregation layer; then directly search for the node matching the core content of the target text from the search range corresponding to the j-1th non-leaf node aggregation layer to obtain the target node corresponding to the j-1th non-leaf node aggregation layer. Wherein, j is a positive integer, 2≤j≤J. It should be noted that for the relevant content of the core content, please refer to the above.

[0158] For example, in some application scenarios, the above step 217 can be specifically as follows: after determining the target node corresponding to the j-1th non-leaf node aggregation layer in the above model tree, first determine the search range corresponding to the j-1th non-leaf node aggregation layer based on the target node corresponding to the j-1th non-leaf node aggregation layer, so that the search range includes at least one child node of the target node corresponding to the j-1th non-leaf node aggregation layer; then directly search for nodes that match at least one key content of the target text from the search range corresponding to the j-1th non-leaf node aggregation layer to obtain the target node corresponding to the j-1th non-leaf node aggregation layer. Wherein, j is a positive integer, 2≤j≤J. It should be noted that the relevant content of the at least one key content can be found above.

[0159] Based on the relevant content of step 217 above, it can be known that for the above model tree, after determining the target node corresponding to the j-1th non-leaf node aggregation layer in the model tree, all child nodes of the target node corresponding to the j-1th non-leaf node aggregation layer (that is, the search range corresponding to the j-th non-leaf node aggregation layer) can be searched for nodes that match the target text to obtain the target node corresponding to the j-th non-leaf node aggregation layer, so that the matching result corresponding to the target text can be determined based on the target node in the future. j is a positive integer, 2≤j≤J, which is conducive to improving search efficiency.

[0160] Step 218: Determine the matching result based on the target nodes corresponding to the J non-leaf node aggregation layers.

[0161] In the present disclosure, for the above model tree, after obtaining the target nodes corresponding to each non-leaf node aggregation layer in the model tree, the matching results corresponding to the above target text can be determined directly based on these target nodes, so that the matching results include the target nodes corresponding to all non-leaf node aggregation layers in the model tree, so that the matching results can represent the categories represented by these target nodes, and further enable the matching results to represent the category affiliation status of the image generation model suitable for image generation processing for the target text.

[0162] Based on the relevant contents of steps 216 to 218 above, it can be known that in some application scenarios, after obtaining the target text input by the user, the category matching the target text can be searched layer by layer in at least two non-leaf node aggregation layers of the model tree to obtain a matching result, so that the matching result can represent the category affiliation status of the image generation model suitable for performing image generation processing on the target text. Among them, for any other non-leaf node aggregation layer in the model tree except the non-leaf node aggregation layer with the most forward arrangement position, the search range required for searching for the target text in the non-leaf node aggregation layer only includes some nodes in the non-leaf node aggregation layer, which is conducive to improving the search efficiency for the non-leaf node aggregation layer, thereby improving the execution efficiency of matching the target text in the model tree.

[0163] Based on the relevant content of step 21 above, it can be known that in some application scenarios, after obtaining the target text input by the user, the categories that match the semantic information carried by the target text can be searched layer by layer from the pre-built model tree to obtain matching results, so that the matching results can represent the category affiliation status of the image generation model suitable for image generation processing for the target text, so that the image generation model suitable for image generation processing for the target text can be determined based on the matching results.

[0164] Step 22: Based on the above matching results, determine the target generation model corresponding to the target text.

[0165] It should be noted that the present disclosure does not limit the implementation method of the above step 22. For example, it can be specifically as follows: after obtaining the matching result corresponding to the above target text, first search the leaf node corresponding to the matching result from the above model tree, so that the candidate generation model represented by the leaf node has the category attribution state described by the matching result; then determine the target generation model corresponding to the target text from the candidate generation model represented by the leaf node (for example, if there is only one leaf node corresponding to the matching result, then directly determine the candidate generation model represented by the leaf node as the target generation model corresponding to the target text; if there are multiple leaf nodes corresponding to the matching result, then select a model from the candidate generation models represented by the multiple leaf nodes in a preset manner as the target generation model corresponding to the target text). The preset manner can be pre-set, for example, the preset manner can be a random selection. For example, the preset manner can be a selection according to a certain pre-set selection rule, etc.

[0166] In fact, in order to better improve the accuracy of model selection, the present disclosure also provides a possible implementation of the above step 22. Under this implementation, the step 22 may specifically include the following steps 221 and 222.

[0167] Step 221: After obtaining the matching result corresponding to the above target text, determine at least one candidate leaf node corresponding to the matching result from the model tree.

[0168] Wherein, candidate leaf node refers to the leaf node existing in the above model tree, with all nodes in the matching result corresponding to the above target text as direct nodes, so that the candidate generation model represented by the candidate leaf node has the category attribution state described by the matching result. That is, for the yth candidate leaf node (for example, node 12 or node 13 shown in Figure 3), the matching result corresponding to the target text may include the parent node of the yth candidate leaf node (for example, node 6 shown in Figure 3), the grandparent node of the yth candidate leaf node (for example, node 3 shown in Figure 3), ... (and so on). y is a positive integer, y≤Y, Y is a positive integer, and Y represents the number of leaf nodes in the above "at least one candidate leaf node".

[0169] Based on the relevant content of step 221 above, it can be known that for some application scenarios, after searching for the matching results corresponding to the target text from the model tree (for example, node 3 and node 6 shown in Figure 3), the leaf nodes with all the nodes in the matching results as direct nodes can be searched from the model tree as candidate leaf nodes corresponding to the matching results, so that the image generation model that is most suitable for image generation processing for the target text can be selected from the candidate generation models represented by these candidate leaf nodes.

[0170] Step 222: Determine the target generation model corresponding to the target text based on the data representing the degree of matching between the target text and the model description information corresponding to each candidate leaf node.

[0171] The model description information corresponding to the yth candidate leaf node is used to describe the characteristics of the candidate generation model represented by the yth candidate leaf node; and the present disclosure does not limit the model description information of the yth candidate leaf node. For example, in some application scenarios, the model description information of the yth candidate leaf node may include the model name corresponding to the yth candidate leaf node and at least one model attribute corresponding to the yth candidate leaf node. The model name corresponding to the yth candidate leaf node is used to uniquely identify the candidate generation model represented by the yth candidate leaf node. The at least one model attribute corresponding to the yth candidate leaf node is used to describe the attribute state of the candidate generation model represented by the yth candidate leaf node; and the present disclosure does not limit the at least one model attribute corresponding to the yth candidate leaf node. For example, in some application scenarios, the at least one model attribute corresponding to the yth candidate leaf node may include one or more attribute tags pre-annotated for the candidate generation model represented by the yth candidate leaf node. y is a positive integer, y≤Y, Y is a positive integer, and Y represents the number of leaf nodes in the above "at least one candidate leaf node".

[0172] In addition, for the yth candidate leaf node above, the matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node is used to describe the matching degree between the target text and the model description information corresponding to the yth candidate leaf node, so that the "matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node" can represent the degree of adaptation between the target text and the candidate generation model represented by the yth candidate leaf node, thereby enabling the "matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node" to represent the possibility of using the candidate generation model represented by the yth candidate leaf node to perform image generation processing on the target text. y is a positive integer, y≤Y, Y is a positive integer, and Y represents the number of leaf nodes in the above "at least one candidate leaf node".

[0173] In addition, the present disclosure does not limit the method for determining the above-mentioned "data representing the degree of matching between the target text and the model description information corresponding to the y-th candidate leaf node". For example, it can be implemented using any existing or future method that can measure the degree of matching between the two pieces of information (for example, by processing with the help of a pre-built language model).

[0174] For example, in some application scenarios, the process of determining the "matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node" includes: calculating the matching degree representation data between the core content of the target text and the model description information corresponding to the yth candidate leaf node, as the matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node. It should be noted that the relevant content of the core content is mentioned above.

[0175] For example, the process of determining the "matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node" includes: calculating the matching degree representation data between at least one key content of the target text and the model description information corresponding to the yth candidate leaf node as the matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node. It should be noted that the relevant content of the at least one key content is described above.

[0176] In addition, the present disclosure does not limit the implementation method of the above step 222. For example, it can be specifically as follows: after determining at least one candidate leaf node corresponding to the above target text from the model tree, based on the data representing the degree of matching between the target text and the model description information corresponding to each candidate leaf node, a model that meets the preset matching conditions is determined from the candidate generation models represented by the at least one candidate leaf node as the target generation model corresponding to the target text. Wherein, the preset matching conditions refer to the conditions satisfied by the candidate generation model finally selected for a text content (for example, the above target text); and the present disclosure does not limit the implementation method of the preset matching conditions. For example, the preset matching conditions can be specifically as follows: the degree of matching between the candidate leaf node representing the target generation model corresponding to the target text and the target text reaches the highest (that is, the highest degree of adaptation).

[0177] Based on the relevant content of step 222 above, it can be known that for some application scenarios, after searching for at least one candidate leaf node for the target text from the model tree, the degree of adaptation between the target text and the candidate generation model represented by each candidate leaf node can be determined based on the model description information of each candidate leaf node; and then the model with the highest degree of adaptation is selected from the candidate generation models represented by the at least one candidate leaf node as the target generation model corresponding to the target text, so that the target generation model can represent the image generation model that is most suitable for image generation processing of the target text.

[0178] Based on the relevant content of steps 221 to 222 above, it can be seen that for some application scenarios, after searching for the matching results corresponding to the target text from the model tree (for example, node 3 and node 6 shown in Figure 3), you can first determine at least one candidate leaf node corresponding to the target text from the model tree based on the matching results; then, based on the target text and the model description information of the candidate generation models represented by these candidate leaf nodes, determine the degree of adaptation between the target text and the candidate generation models represented by each candidate leaf node; then, select the model with the highest degree of adaptation from the candidate generation models represented by the at least one candidate leaf node as the target generation model corresponding to the target text, so that the target generation model can represent the image generation model that is most suitable for image generation processing for the target text.

[0179] Based on the relevant content of S2 above, it can be seen that in some application scenarios, after obtaining the target text input by the user, a set of candidate models that are relatively matched with the target text can be searched from a pre-constructed model tree (for example, the idea tree shown in Figure 3) (for example, the image generation model represented by each node in the candidate set shown in Figure 3); then, based on the core content of the target text (for example, the entire text content, core content or at least one key content, etc.) and the model description information of the candidate generation models represented by these candidate leaf nodes, the image generation model that is most suitable for image generation processing for the target text is selected from the candidate model set as the target generation model corresponding to the target text.

[0180] S3: Use the target generation model to generate image data corresponding to the target text.

[0181] In the present disclosure, after obtaining the target generation model corresponding to the target text described above, the target generation model can be directly used to generate image data corresponding to the target text (for example, the target text is directly input into the target generation model so that the target generation model performs image generation processing on the target text, and obtains and outputs image data corresponding to the target text). Since the target generation model is more suitable for performing image generation processing on the target text, the image data generated by the target generation model for the target text can be more consistent with the semantic information expressed by the target text, which is conducive to improving the image generation effect.

[0182] In fact, in order to better improve the image generation effect, the present disclosure also provides a possible implementation of the above S3. In this implementation, S3 may specifically include the following steps 31 and 32.

[0183] Step 31: After obtaining the target generation model corresponding to the target text, the target text is updated according to the text format constraint information corresponding to the target generation model to obtain an updated text.

[0184] The text format constraint information corresponding to the target generation model is used to indicate the text format for which the target generation model is more suitable for image generation processing.

[0185] In addition, the present disclosure does not limit the implementation method of the above-mentioned "text format constraint information corresponding to the target generation model". For example, in some application scenarios, the "text format constraint information corresponding to the target generation model" may include constraint description content under one or more format dimensions. It should be noted that the present disclosure does not limit the implementation method of the one or more format dimensions. For example, it can be set according to the actual application scenario. In addition, the present disclosure does not limit the implementation method of the constraint description content of the one or more format dimensions. For example, it can be set according to the actual application scenario.

[0186] For example, in some application scenarios, the "text format constraint information corresponding to the target generation model" mentioned above may include a processed text sample corresponding to the target generation model. The processed text sample refers to a text example pre-set for the target generation model and suitable for image processing using the target generation model, so that the processed text sample can indicate which text format the target generation model is more suitable for image generation processing.

[0187] In addition, the present disclosure does not limit the number of the above-mentioned processed text samples. For example, in some application scenarios, the above-mentioned "text format constraint information corresponding to the target generation model" may include one or more processed text samples corresponding to the target generation model, so that the "text format constraint information corresponding to the target generation model" can indicate as comprehensively and accurately as possible what format of text the target generation model is more suitable for image generation processing.

[0188] In addition, the present disclosure does not limit the method for obtaining the above-mentioned processed text sample. For example, the processed text sample can be pre-set. For another example, the processed text sample can be automatically determined based on the training data of the above-mentioned target generation model.

[0189] The updated text refers to the text obtained by updating the target text according to the text format constraint information corresponding to the target generation model above, so that the semantic information expressed by the updated text is consistent with the semantic information expressed by the target text, and the updated text satisfies the text format described by the text format constraint information, so that the updated text can express the semantic information expressed by the target text on the premise of satisfying the text format described by the text format constraint information, and further enables the updated text to express the semantic information expressed by the target text according to the text format requirements of the target generation model.

[0190] It should be noted that the present disclosure does not limit the implementation method of the "update process". For example, in some application scenarios, the "update process" can be implemented by rewriting. Based on this, it can be seen that in one possible implementation method, the above step 31 can be specifically as follows: after obtaining the target generation model corresponding to the above target text, the target text is rewritten according to the text format constraint information corresponding to the target generation model to obtain the above updated text, so that the updated text can represent the text obtained by rewriting the target text according to the text format constraint information corresponding to the target generation model.

[0191] Based on the relevant content of step 31 above, it can be known that for some application scenarios, after obtaining the target generation model corresponding to the target text, the target text can be updated according to the text format constraint information corresponding to the target generation model to obtain an updated text, so that the updated text can express the semantic information expressed by the target text on the premise of satisfying the text format described by the text format constraint information, thereby enabling the updated text to express the semantic information expressed by the target text in accordance with the text format requirements of the target generation model.

[0192] Step 32: Use the above target generation model to perform image generation processing on the updated text to obtain image data corresponding to the target text.

[0193] In the present disclosure, after obtaining the updated text corresponding to the target text above, the updated text can be directly input into the target generation model corresponding to the target text, so that the target generation model can perform image generation processing on the updated text and obtain and output image data corresponding to the target text. In particular, because the updated text can express the semantic information expressed by the target text according to the text format requirements of the target generation model, the updated text is more compatible with the target generation model, so that the image data generated for the updated text using the target generation model can better conform to the semantic information expressed by the target text, which is conducive to improving the image generation effect.

[0194] Based on the relevant contents of S1 to S3 above, it can be known that after obtaining the target text, the pre-constructed model tree is first used to determine the target generation model corresponding to the target text from at least two candidate generation models, so that the target generation model can represent the model that exists in the at least two candidate generation models and is more suitable for image generation processing for the target text; then the target generation model is used to generate image data corresponding to the target text, so that the image data can be as consistent as possible with the semantic information expressed by the target text, thereby achieving the purpose of generating image data from text.

[0195] In addition, since the above model tree is used to describe the category affiliation status of at least two candidate generation models above, the model tree can better represent the characteristics of different candidate generation models, so that the target generation model determined subsequently using the model tree for a certain text (for example, the target text above) is more suitable for image generation processing for the text, which can effectively avoid defects caused by the incompatibility between the text and the image generation model (for example, defects such as poor image generation effect), thereby helping to improve the image generation effect.

[0196] In addition, the present disclosure does not limit the execution subject of the image generation method provided in the embodiments of the present disclosure. For example, the image generation method provided in the embodiments of the present disclosure can be applied to a terminal device or a server. For another example, the image generation method provided in the embodiments of the present disclosure can also be implemented with the help of a data interaction process between a terminal device and a server. The terminal device can be a smart phone, a computer, a personal digital assistant (PDA), a tablet computer, etc. The server can be a standalone server, a cluster server, or a cloud server.

[0197] Based on the image generation method provided in the embodiments of the present disclosure, the embodiments of the present disclosure also provide an image generation device, which will be explained and illustrated below in conjunction with Figure 4. Figure 4 is a schematic structural diagram of the image generation device provided in the embodiments of the present disclosure. It should be noted that for the technical details of the image generation device provided in the embodiments of the present disclosure, please refer to the relevant content of the image generation method above.

[0198] As shown in FIG4 , an image generating apparatus 400 provided in an embodiment of the present disclosure includes:

[0199] An acquisition unit 401 is used to acquire a target text;

[0200] A determination unit 402 is configured to determine a target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is configured to describe the category attribution status of the at least two candidate generation models;

[0201] The generating unit 403 is configured to generate image data corresponding to the target text using the target generation model.

[0202] In one possible implementation, different leaf nodes in the model tree are used to represent different candidate generation models; for any leaf node in the model tree, the direct node of the leaf node is used to represent the category to which the candidate generation model represented by the leaf node belongs.

[0203] In a possible implementation manner, the determining unit 402 includes:

[0204] A first search subunit is configured to search the model tree for non-leaf nodes that match the target text to obtain a matching result;

[0205] The first determination subunit is used to determine the target generation model corresponding to the target text based on the matching result.

[0206] In a possible implementation manner, the first search subunit includes:

[0207] A core content extraction subunit is used to perform core content extraction processing on the target text to obtain the core content;

[0208] The second search subunit is used to search the model tree for non-leaf nodes that match the core content to obtain matching results.

[0209] In one possible implementation, the core content extraction sub-unit is specifically used to: determine a target style that matches the target text from extracted text styles corresponding to at least two core content extraction algorithms; and perform core content extraction processing on the target text using the core content extraction algorithm corresponding to the target style to obtain the core content.

[0210] In one possible implementation, the second search sub-unit is specifically used to: perform key content extraction processing on the core content to obtain at least one key content; search for non-leaf nodes that match the at least one key content from the model tree to obtain a matching result.

[0211] In a possible implementation manner, the model tree includes J non-leaf node aggregation layers arranged in sequence, where J is a positive integer; each non-leaf node aggregation layer includes at least two non-leaf nodes;

[0212] The first search sub-unit is specifically used to: search for nodes matching the target text from the first non-leaf node aggregation layer, and obtain the target node corresponding to the first non-leaf node aggregation layer; search for nodes matching the target text from the search range corresponding to the j-th non-leaf node aggregation layer, and obtain the target node corresponding to the j-th non-leaf node aggregation layer; the search range corresponding to the j-th non-leaf node aggregation layer is determined based on the target node corresponding to the j-1-th non-leaf node aggregation layer; j is a positive integer, 2≤j≤J; the matching result is determined based on the target nodes corresponding to the J non-leaf node aggregation layers.

[0213] In a possible implementation manner, the parent node of each node in the search range corresponding to the j-th non-leaf node aggregation layer is the target node corresponding to the j-1-th non-leaf node aggregation layer.

[0214] In one possible implementation, the first determination subunit is specifically used to: determine at least one candidate leaf node corresponding to the matching result from the model tree; and determine the target generation model corresponding to the target text based on the matching degree representation data between the target text and the model description information corresponding to each candidate leaf node.

[0215] In a possible implementation manner, the model description information includes a model name and at least one model attribute.

[0216] In a possible implementation, the model tree is constructed based on model description information of the at least two candidate generation models, where the model description information includes a model name and at least one model attribute.

[0217] In one possible implementation, the model tree includes at least one non-leaf node aggregation layer and a leaf node aggregation layer; different nodes in the leaf node aggregation layer are used to represent the model names of different candidate generation models; the categories represented by the nodes in the different non-leaf node aggregation layers are determined based on the classification results of the at least two candidate generation models under different attribute dimensions.

[0218] In one possible implementation, the generation unit 403 is specifically used to: update the target text based on the text format constraint information corresponding to the target generation model to obtain updated text; and use the target generation model to perform image generation processing on the updated text to obtain image data corresponding to the target text.

[0219] In a possible implementation manner, the text format constraint information corresponding to the target generation model includes a processed text sample corresponding to the target generation model.

[0220] Based on the relevant content of the above-mentioned image generation device 400, it can be known that for the image generation device 400 provided by the embodiment of the present disclosure, after obtaining the target text, the pre-constructed model tree is first used to determine the target generation model corresponding to the target text from at least two candidate generation models, so that the target generation model can represent the model that exists in the at least two candidate generation models and is more suitable for image generation processing for the target text; and then the target generation model is used to generate image data corresponding to the target text, so that the image data can be as consistent as possible with the semantic information expressed by the target text, thereby achieving the purpose of generating image data from text.

[0221] Based on the image generation method provided in the embodiments of the present disclosure, the embodiments of the present disclosure also provide an image generation system, which will be explained and illustrated below in conjunction with Figure 5. Figure 5 is a schematic diagram of the structure of the image generation system provided in the embodiments of the present disclosure. It should be noted that for the technical details of the image generation system provided in the embodiments of the present disclosure, please refer to the relevant content of the image generation method above.

[0222] As shown in FIG5 , an image generation system 500 provided in an embodiment of the present disclosure includes a controller 501 and at least two candidate generation models. The controller 501 is configured to execute any embodiment of the image generation method provided in the present disclosure. For ease of understanding, the following description is provided with examples.

[0223] As an example, in one possible implementation, the controller 501 is used to obtain the target text; the controller 501 is also used to use a pre-built model tree to determine the target generation model corresponding to the target text from at least two candidate generation models; the model tree is used to describe the category attribution status of the at least two candidate generation models; the controller 501 is also used to call the target generation model to obtain image data corresponding to the target text; the target generation model is used to generate image data corresponding to the target text.

[0224] In a possible implementation manner, the controller 501 is a language model.

[0225] Based on the relevant content of the image generation system 500 above, it can be known that for the image generation system 500 provided by the embodiment of the present disclosure, after obtaining the target text, the pre-constructed model tree is first used to determine the target generation model corresponding to the target text from at least two candidate generation models, so that the target generation model can represent the model that exists in the at least two candidate generation models and is more suitable for image generation processing for the target text; then, by calling the target generation model, the image data corresponding to the target text generated by the target generation model is obtained, so that the image data can be as consistent as possible with the semantic information expressed by the target text, thereby achieving the purpose of generating image data from text.

[0226] It can be seen that for the image generation system provided by the present disclosure, the image generation system can generate high-quality images by searching for a generation model that best matches a certain text content, so that the image generation system can have the advantages of high versatility, high practicality and greater convenience. In addition, the image generation system can be composed of a language model and a plurality of candidate generation models, so that the language model in the image generation system can act as a core controller to maintain the entire workflow of the system (for example, core content extraction, construction and search model thought tree, execution image generation and other processes), so that the image generation system can balance the advantages of general image generation models and image generation models in certain specific fields with the help of the language model, and automatically find the best matching model for any text content (for example, text content input by the user) to perform image generation processing, so that the system can more flexibly achieve the purpose of generating high-quality images from text. In addition, the image generation system can more accurately extract the generation target described by any text content with the help of multiple core content extraction algorithms, which is conducive to improving the image generation effect. In addition, the image generation system can determine the model that best matches the text content by first searching the text content layer by layer in the model tree and then performing a refined search based on multiple attributes of the model. This is conducive to improving the accuracy of model matching and thus improving the image generation effect.

[0227] In addition, an embodiment of the present disclosure also provides an electronic device, which includes a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes any implementation of the image generation method provided by the embodiment of the present disclosure.

[0228] Referring to FIG6 , a schematic diagram of the structure of an electronic device 600 suitable for implementing embodiments of the present disclosure is shown. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG6 is merely an example and should not limit the functionality or scope of use of embodiments of the present disclosure.

[0229] As shown in Figure 6, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. Various programs and data required for the operation of electronic device 600 are also stored in RAM 603. Processing device 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to bus 604.

[0230] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although FIG. 6 shows the electronic device 600 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0231] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0232] The electronic device provided by the embodiment of the present disclosure and the method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0233] The embodiments of the present disclosure further provide a computer-readable medium, in which instructions or computer programs are stored. When the instructions or computer programs are executed on a device, the device executes any implementation of the image generation method provided by the embodiments of the present disclosure.

[0234] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0235] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0236] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0237] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device can perform the method.

[0238] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0239] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0240] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit / module does not, in some cases, limit the unit itself.

[0241] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0242] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0243] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0244] It should be understood that in the present disclosure, "at least one (item)" refers to one or more, and "plurality" refers to two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0245] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0246] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0247] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present disclosure. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to the embodiments shown herein, but is intended to be construed in the widest manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating an image, the method comprising: Get the target text; Determine a target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; The model tree is used to describe the category attribution status of the at least two candidate generation models; The target generation model is used to generate image data corresponding to the target text.

2. The method according to claim 1, wherein: Different leaf nodes in the model tree are used to represent different candidate generation models; For any leaf node in the model tree, the direct node of the leaf node is used to represent the category to which the candidate generation model represented by the leaf node belongs.

3. The method according to claim 2, wherein: The process of determining the target generation model includes: Searching the model tree for non-leaf nodes that match the target text to obtain a matching result; According to the matching result, a target generation model corresponding to the target text is determined.

4. The method according to claim 3, wherein: The process of determining the matching result includes: Performing core content extraction processing on the target text to obtain core content; The non-leaf nodes matching the core content are searched from the model tree to obtain matching results.

5. The method according to claim 4, wherein: The process of determining the core content includes: Determining a target style matching the target text from among the extracted text styles corresponding to at least two core content extraction algorithms; The core content extraction algorithm corresponding to the target style is used to perform core content extraction processing on the target text to obtain the core content.

6. The method according to claim 4, wherein: Before searching the model tree for non-leaf nodes matching the core content and obtaining a matching result, the method further includes: Performing key content extraction processing on the core content to obtain at least one key content; The step of searching the model tree for non-leaf nodes matching the core content to obtain matching results includes: The model tree is searched for non-leaf nodes matching the at least one key content to obtain a matching result.

7. The method according to claim 3, wherein: The model tree includes J non-leaf node aggregation layers arranged in sequence, where J is a positive integer; Each of the non-leaf node aggregation layers includes at least two non-leaf nodes; The process of determining the matching result includes: Searching for nodes matching the target text from the first non-leaf node aggregation layer to obtain a target node corresponding to the first non-leaf node aggregation layer; Searching for nodes matching the target text from the search range corresponding to the j-th non-leaf node aggregation layer to obtain the target node corresponding to the j-th non-leaf node aggregation layer; the search range corresponding to the j-th non-leaf node aggregation layer is determined based on the target node corresponding to the j-1-th non-leaf node aggregation layer; j is a positive integer, 2≤j≤J; The matching result is determined according to the target nodes corresponding to the J non-leaf node aggregation layers.

8. The method according to claim 7, wherein: The parent node of each node in the search range corresponding to the j-th non-leaf node aggregation layer is the corresponding target node of the j-1-th non-leaf node aggregation layer.

9. The method according to claim 3, wherein: Determining the target generation model corresponding to the target text according to the matching result includes: Determine at least one candidate leaf node corresponding to the matching result from the model tree; The target generation model corresponding to the target text is determined based on the data representing the degree of matching between the target text and the model description information corresponding to each of the candidate leaf nodes.

10. The method according to claim 9, wherein: The model description information includes a model name and at least one model attribute.

11. The method according to claim 1, wherein: The model tree is constructed based on model description information of the at least two candidate generation models, wherein the model description information includes a model name and at least one model attribute; and / or, The model tree includes at least one non-leaf node aggregation layer and a leaf node aggregation layer; different nodes in the leaf node aggregation layer are used to represent the model names of different candidate generation models; the categories represented by the nodes in the different non-leaf node aggregation layers are determined based on the classification results of the at least two candidate generation models under different attribute dimensions.

12. The method according to claim 1, wherein: The process of determining the image data corresponding to the target text includes: According to the text format constraint information corresponding to the target generation model, the target text is updated to obtain an updated text; The target generation model is used to perform image generation processing on the updated text to obtain image data corresponding to the target text.

13. The method according to claim 12, wherein: The text format constraint information corresponding to the target generation model includes processed text samples corresponding to the target generation model.

14. An image generating device, comprising: An acquisition unit, used for acquiring a target text; A determination unit, configured to determine a target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is configured to describe the category attribution status of the at least two candidate generation models; A generating unit is used to generate image data corresponding to the target text by using the target generation model.

15. An image generation system, the image generation system comprising a controller and at least two candidate generation models; The controller is used to obtain the target text; The controller is further used to determine the target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is used to describe the category attribution status of the at least two candidate generation models; The controller is further used to call the target generation model to obtain image data corresponding to the target text; the target generation model is used to generate image data corresponding to the target text.

16. The system of claim 15, wherein: The controller is a language model.

17. An electronic device, comprising: Processor and memory; The memory is used to store instructions or computer programs; The processor is used to execute the instructions or computer programs in the memory so that the electronic device executes the method according to any one of claims 1 to 13.

18. A computer-readable medium storing instructions or computer programs, which, when executed on a device, enable the device to execute the method according to any one of claims 1 to 13.

Images