Image generation method, device and system, electronic equipment and computer readable medium

Abstract

The invention discloses an image generation method, device and system, electronic equipment and a computer readable medium, and the method comprises the steps: after obtaining a target text, determining a target generation model corresponding to the target text from at least two candidate generation models through employing a pre-constructed model tree, the target generation model can represent models which exist in the at least two candidate generation models and are suitable for carrying out image generation processing on the target text; and generating image data corresponding to the target text by using the target generation model, so that the image data can accord with semantic information expressed by the target text as much as possible, and the purpose of generating the image data from the text can be achieved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to an image generation method, device, system, electronic device, and computer-readable medium. Background Art

[0002] In some application scenarios (e.g., image generation scenarios, etc.), it may be necessary to perform image generation processing based on text content provided by the user (e.g., the text content of "a black puppy running") so that the generated image data can represent the semantic information expressed by the text content.

[0003] However, how to realize the requirements described in the previous paragraph is a technical problem that needs to be solved urgently. Summary of the invention

[0004] The present application provides an image generation method, device, system, electronic device, and computer-readable medium, which can achieve the purpose of generating image data from text.

[0005] In order to achieve the above objectives, the technical solutions provided by this application are as follows:

[0006] The present application provides an image generation method, the method comprising:

[0007] Get the target text;

[0008] Determine the target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is used to describe the category attribution status of the at least two candidate generation models;

[0009] The target generation model is used to generate image data corresponding to the target text.

[0010] In a possible implementation manner, different leaf nodes in the model tree are used to represent different candidate generation models;

[0011] For any leaf node in the model tree, the direct node of the leaf node is used to represent the category to which the candidate generation model represented by the leaf node belongs.

[0012] In a possible implementation manner, the process of determining the target generation model includes:

[0013] Searching the model tree for non-leaf nodes that match the target text to obtain a matching result;

[0014] According to the matching result, a target generation model corresponding to the target text is determined.

[0015] In a possible implementation manner, the process of determining the matching result includes:

[0016] Performing core content extraction processing on the target text to obtain the core content;

[0017] The non-leaf nodes matching the core content are searched from the model tree to obtain matching results.

[0018] In a possible implementation manner, the process of determining the core content includes:

[0019] Determining a target style matching the target text from among the extracted text styles corresponding to at least two core content extraction algorithms;

[0020] The core content extraction algorithm corresponding to the target style is used to perform core content extraction processing on the target text to obtain the core content.

[0021] In a possible implementation manner, before searching the model tree for non-leaf nodes matching the core content to obtain a matching result, the method further includes:

[0022] Performing key content extraction processing on the core content to obtain at least one key content;

[0023] The step of searching the model tree for non-leaf nodes matching the core content to obtain matching results includes:

[0024] The model tree is searched for non-leaf nodes matching the at least one key content to obtain a matching result.

[0025] In a possible implementation manner, the model tree includes J non-leaf node aggregation layers arranged in sequence, where J is a positive integer; each of the non-leaf node aggregation layers includes at least two non-leaf nodes;

[0026] The process of determining the matching result includes:

[0027] Searching for nodes matching the target text from the first non-leaf node aggregation layer to obtain a target node corresponding to the first non-leaf node aggregation layer;

[0028] Searching for nodes matching the target text from the search range corresponding to the j-th non-leaf node aggregation layer to obtain the target node corresponding to the j-th non-leaf node aggregation layer; the search range corresponding to the j-th non-leaf node aggregation layer is determined based on the target node corresponding to the j-1-th non-leaf node aggregation layer; j is a positive integer, 2≤j≤J;

[0029] The matching result is determined according to the target nodes corresponding to the J non-leaf node aggregation layers.

[0030] In a possible implementation manner, the parent node of each node in the search range corresponding to the j-th non-leaf node aggregation layer is the corresponding target node of the j-1-th non-leaf node aggregation layer.

[0031] In a possible implementation manner, determining the target generation model corresponding to the target text according to the matching result includes:

[0032] Determine at least one candidate leaf node corresponding to the matching result from the model tree;

[0033] The target generation model corresponding to the target text is determined based on the data representing the degree of matching between the target text and the model description information corresponding to each of the candidate leaf nodes.

[0034] In a possible implementation manner, the model description information includes a model name and at least one model attribute.

[0035] In a possible implementation manner, the model tree is constructed based on model description information of the at least two candidate generation models, and the model description information includes a model name and at least one model attribute.

[0036] In one possible implementation, the model tree includes at least one non-leaf node aggregation layer and a leaf node aggregation layer; different nodes in the leaf node aggregation layer are used to represent the model names of different candidate generation models; the categories represented by the nodes in the different non-leaf node aggregation layers are determined based on the classification results of the at least two candidate generation models under different attribute dimensions.

[0037] In a possible implementation manner, the process of determining the image data corresponding to the target text includes:

[0038] According to the text format constraint information corresponding to the target generation model, the target text is updated to obtain an updated text;

[0039] The target generation model is used to perform image generation processing on the updated text to obtain image data corresponding to the target text.

[0040] In a possible implementation manner, the text format constraint information corresponding to the target generation model includes a processed text sample corresponding to the target generation model.

[0041] The present application provides an image generation device, comprising:

[0042] An acquisition unit, used for acquiring a target text;

[0043] A determination unit, configured to determine a target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is configured to describe the category attribution status of the at least two candidate generation models;

[0044] A generating unit is used to generate image data corresponding to the target text by using the target generation model.

[0045] The present application provides an image generation system, the image generation system comprising a controller and at least two candidate generation models;

[0046] The controller is used to obtain the target text;

[0047] The controller is further used to determine the target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is used to describe the category attribution status of the at least two candidate generation models;

[0048] The controller is further used to call the target generation model to obtain image data corresponding to the target text; the target generation model is used to generate image data corresponding to the target text.

[0049] In a possible implementation manner, the controller is a language model.

[0050] The present application provides an electronic device, the device comprising: a processor and a memory;

[0051] The memory is used to store instructions or computer programs;

[0052] The processor is used to execute the instructions or computer programs in the memory so that the electronic device executes the image generation method provided in the present application.

[0053] The present application provides a computer-readable medium, characterized in that instructions or computer programs are stored in the computer-readable medium, and when the instructions or computer programs are executed on a device, the device executes the image generation method provided by the present application.

[0054] The present application provides a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program codes for executing the image generation method provided by the present application.

[0055] Compared with the related art, this application has at least the following advantages:

[0056] In the technical solution provided by the present application, after obtaining the target text, a pre-constructed model tree is first used to determine a target generation model corresponding to the target text from at least two candidate generation models, so that the target generation model can represent a model that exists in the at least two candidate generation models and is more suitable for image generation processing for the target text; and then the target generation model is used to generate image data corresponding to the target text, so that the image data can be as consistent as possible with the semantic information expressed by the target text, thereby achieving the purpose of generating image data from text.

[0057] In addition, because the above model tree is used to describe the category affiliation status of at least two candidate generation models above, the model tree can better represent the characteristics of different candidate generation models, so that the target generation model determined by the subsequent use of the model tree for a certain text (for example, the target text above) is more suitable for image generation processing for the text, which can effectively avoid defects caused by the incompatibility between the text and the image generation model (for example, defects such as poor image generation effect), thereby helping to improve the image generation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 A flowchart of an image generation method provided in an embodiment of the present application;

[0060] Figure 2 A schematic diagram of an image generation process provided in an embodiment of the present application;

[0061] Figure 3 A schematic diagram of a thought tree provided in an embodiment of the present application;

[0062] Figure 4 A schematic diagram of the structure of an image generating device provided in an embodiment of the present application;

[0063] Figure 5 A schematic diagram of the structure of an image generation system provided in an embodiment of the present application;

[0064] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0065] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0066] In order to better understand the technical solution provided by the present application, the image generation method provided by the present application is described below with reference to some drawings. Figure 1 As shown, the image generation method provided in the embodiment of the present application includes S1-S3. Figure 1 A flowchart of an image generation method provided in an embodiment of the present application.

[0067] S1: Get the target text.

[0068] The target text refers to the text required for image generation processing; and the present application does not limit the target text. For example, in some application scenarios, the target text may refer to the text content input by the user with the help of some input device (for example, Figure 2 The text “Generate an image of a dog” shown above).

[0069] In addition, the present application does not limit the method for obtaining the target text above. For example, in some application scenarios, the target text can be obtained from the information input by the user. For another example, in some application scenarios, the target text can be read from some storage space. For another example, in some application scenarios, the target text can refer to the text content with image generation processing requirements involved in the execution of other tasks.

[0070] S2: Determine a target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is used to describe the category attribution status of the at least two candidate generation models.

[0071] The candidate generation model refers to a pre-built image generation model with good image generation performance. Figure 2 For model 1 or model 2) shown in the figure, the candidate generation model can be used to perform image generation processing on the input data of the candidate generation model.

[0072] In addition, in some application scenarios, for the above “at least two candidate generation models”, different candidate generation models are suitable for implementing image generation processing in different application scenarios, so that different candidate generation models are suitable for image generation processing for different types of text.

[0073] In addition, the present application does not limit the implementation method of the candidate generation model above. For example, in some application scenarios, the candidate generation model may adopt a diffusion model (for example, Figure 2 The diffusion model shown in FIG.

[0074] The model tree is used to express the association relationship (e.g., whether they belong to the same category under a certain attribute dimension, etc.) between some pre-built image generation models (e.g., the "at least two candidate generation models" mentioned above) in the form of a tree structure (e.g., a structure such as an idea tree). The image generation model is used to perform image generation processing on the input data of the image generation model. It can be seen that in some application scenarios, the model tree can be used to describe the category attribution status presented by the at least two candidate generation models mentioned above under some attribute dimensions (e.g., attribute dimensions such as subject and style).

[0075] Based on the above content, it can be known that for the above model tree, if the model tree is constructed based on the relevant information of at least two candidate generation models (for example, information such as model attributes and model names), then the model tree can be used to describe the category attribution status of the at least two candidate generation models. Among them, the category attribution status of the i-th candidate generation model refers to the category attribution status presented by the i-th candidate generation model under at least one attribute dimension, i is a positive integer, i≤I, I is a positive integer, and I represents the number of models in the at least two candidate generation models.

[0076] In addition, for the "at least one attribute dimension" in the above paragraph, the "at least one attribute dimension" is some model classification dimensions obtained by refining the model attributes of at least two candidate generation models mentioned above; and the present application does not limit the "at least one attribute dimension", for example, it may include dimensions such as subject, style, and language. Among them, the model attributes of the i-th candidate generation model are used to represent the characteristics of the i-th candidate generation model; and the present application does not limit the model attributes of the i-th candidate generation model, for example, it may include at least one model attribute label pre-annotated for the i-th candidate generation model (for example, the label "animals as the subject", the label "ink painting style", the label "Chinese comics", etc.).

[0077] It should be noted that this application does not limit the determination method of the "at least one attribute dimension" in the above paragraph. For example, in some application scenarios, the "at least one attribute dimension" can be set in advance according to the actual application scenario. For another example, in some application scenarios, in order to better improve flexibility, the "at least one attribute dimension" can be automatically extracted from the model attributes of the at least two candidate generation models.

[0078] In addition, the present application does not limit the implementation of the above model tree. For example, it can be implemented by means of an idea tree (for example, Figure 2 or Figure 3 The idea tree shown in the figure is implemented.

[0079] Also, for the above model tree, the model tree may include at least one non-leaf node aggregation layer (e.g., Figure 2 or Figure 3 The idea tree shown in the figure has all layers except the first and last layers) and a leaf node aggregation layer (e.g. Figure 2 or Figure 3 The non-leaf node refers to a node in the model tree that has at least one child node (e.g., Figure 3 The non-leaf node aggregation layer refers to a layer for aggregating and displaying at least two non-leaf nodes (for example, Figure 3 The second or third layer in the idea tree shown). The jth non-leaf node aggregation layer refers to the non-leaf node aggregation layer that exists in the model tree and is in the jth arrangement position, so that the jth non-leaf node aggregation layer is used to represent the classification status presented by the at least two candidate generation models above under the jth attribute dimension, so that each node existing in the jth non-leaf node aggregation layer can respectively represent the category to which one or more models of the at least two candidate generation models belong under the jth attribute dimension, j is a positive integer, j is less than or equal to J, J is a positive integer, and J represents the number of non-leaf node aggregation layers in the model tree. The leaf node refers to a node that does not have child nodes in the model tree (for example, Figure 3 The leaf node aggregation layer refers to a layer used to aggregate and display leaf nodes (e.g., Figure 3 The 4th layer in the idea tree shown in FIG. 4 ), and different nodes in the leaf node aggregation layer are used to represent the model names of different candidate generation models (for example, Figure 3 The node 8 shown is used to represent the model name "Model 3". Figure 3The node 9 shown is used to represent the model name "Model 4", etc.), so that different nodes in the leaf node aggregation layer can be used to represent different candidate generation models, so that the leaf node aggregation layer can be used to represent the at least two candidate generation models, and then the model tree can describe the category affiliation status of the at least two candidate generation models.

[0080] Based on the above content, we can know that in some application scenarios, for the above model tree (for example, Figure 3 For example, the model tree may include K layers (e.g., Figure 3 4 layers as shown), K is a positive integer. The first layer of the model tree is used to record the root node (for example, Figure 3 Node 1 shown). The k-th layer of the model tree is used to record the classification status presented by the at least two candidate generation models above under the k-1th attribute dimension, so that the k-th layer can represent the k-1th non-leaf node aggregation layer existing in the model tree, so that the category name represented by each node in the k-th layer can represent the category to which one or more models of the at least two candidate generation models above belong under the k-1th attribute dimension, k is a positive integer, 2≤k≤K-1. The K-th layer of the model tree is used to record the leaf nodes corresponding to the at least two candidate generation models, so that the model names represented by each leaf node existing in the K-th layer respectively identify each candidate generation model, so that different nodes in the K-th layer respectively represent different candidate generation models.

[0081] Based on the above content, it can be known that in a possible implementation, for the above model tree, different leaf nodes in the model tree are used to represent different candidate generation models; and for any leaf node in the model tree, the direct node of the leaf node is used to represent the category to which the candidate generation model represented by the leaf node belongs. Among them, the direct node of the qth leaf node refers to the node existing in the model tree and having a certain direct relationship with the qth leaf node (for example, a father-child relationship, a grandparent-grandchild relationship, etc.), so that the direct node of the qth leaf node can represent the non-leaf node that exists in the model tree and appears in the shortest path with the qth leaf node as the starting point and the root node of the model tree as the end point. q is a positive integer, q≤Q, Q is a positive integer, and Q represents the number of leaf nodes in the model tree.

[0082] In addition, the present application does not limit the implementation method of the direct nodes of the qth leaf node above. For example, in some application scenarios, the direct nodes of the qth leaf node may include all non-leaf nodes in the shortest path corresponding to the qth leaf node. The shortest path corresponding to the qth leaf node refers to the shortest path existing in the model tree above, with the qth leaf node as the starting point and the root node of the model tree as the end point.

[0083] It can be seen that in a possible implementation mode, for the qth leaf node in the above model tree (for example, Figure 3 For example, for the node 12 shown in FIG. 1 , the shortest path corresponding to the qth leaf node can be the qth leaf node → the parent node of the qth leaf node → the grandparent node of the qth leaf node → ... (for example, Figure 3 The path of node 12→node 6→node 3→node 1 shown in FIG. 1 ); and the direct node of the qth leaf node may specifically include the parent node of the qth leaf node (for example, Figure 3 The node 6 shown as the parent node of the node 12), the grandparent node of the qth leaf node (that is, the parent node of the parent node of the qth leaf node, for example, Figure 3 The node 3 shown as the grandparent node of the node 12), the great-grandfather node of the qth leaf node (that is, the parent node of the grandparent node of the qth leaf node, for example, Figure 3 Node 1) shown as the great-grandfather node of node 12), ... (and so on).

[0084] For example, in some application scenarios, the direct nodes of the qth leaf node above may include all non-leaf nodes except the root node in the shortest path corresponding to the qth leaf node. Figure 3 When the node is 12 as shown, the shortest path corresponding to the qth leaf node can be node 12→node 6→node 3→node 1; and the direct nodes of the qth leaf node can specifically include the node 6 and the node 3.

[0085] Based on the relevant content of the direct nodes of the qth leaf node in the above text, it can be known that for the qth leaf node, the direct nodes of the qth leaf node can include a node in each non-leaf node aggregation layer in the above model tree that appears in the shortest path corresponding to the qth leaf node, so that the different direct nodes of the qth leaf node can indicate the category to which the candidate generation model represented by the qth leaf node belongs under different attribute dimensions, so that all the direct nodes of the qth leaf node can indicate the category belonging status presented by the candidate generation model represented by the qth leaf node under at least one attribute dimension. It can be seen that when the model tree includes J non-leaf node aggregation layers, the direct nodes of the qth leaf node can include the direct nodes corresponding to the qth leaf node in the first non-leaf node aggregation layer, the direct nodes corresponding to the qth leaf node in the second non-leaf node aggregation layer, ... (and so on), and the direct nodes corresponding to the qth leaf node in the Jth non-leaf node aggregation layer. Among them, the direct node corresponding to the qth leaf node in the jth non-leaf node aggregation layer refers to the node passed by the shortest path corresponding to the qth leaf node in the jth non-leaf node aggregation layer, so that the "direct node corresponding to the qth leaf node in the jth non-leaf node aggregation layer" can represent the category of the candidate generation model represented by the qth leaf node under the jth attribute dimension, j is a positive integer, j≤J, and J is a positive integer.

[0086] Based on the relevant content of the model tree above, it can be known that in some application scenarios, for the model tree, the leaf nodes in the model tree can be used to represent a candidate generation model, and the non-leaf nodes in the model tree except the root node can be used to represent the category to which a candidate generation model belongs under a certain attribute dimension. Based on this, it can be known that in a possible implementation, in order to meet the above requirements, for any leaf node in the model tree, the model identifier of a candidate generation model (for example, identification information such as model name) can be used as the node name of the leaf node, so that the leaf node can be used to represent the model identifier of the candidate generation model, so that the leaf node can represent the candidate generation model; and, for any non-leaf node in the model tree except the root node, the category identifier (for example, category name) of the category to which a candidate generation model belongs under the corresponding attribute dimension can be used as the node name of the non-leaf node, so that the non-leaf node is used to represent the category identifier of the category to which the candidate generation model belongs under the corresponding attribute dimension, so that the non-leaf node can represent the category to which the candidate generation model belongs under the corresponding attribute dimension.

[0087] Based on the above content, it can be known that in a possible implementation, the above model tree may include at least one non-leaf node aggregation layer (for example, Figure 3 The second and third layers of the idea tree shown in the figure) and the leaf node aggregation layer (for example, Figure 3 The fourth layer of the idea tree shown in FIG. 1 ). Different nodes in the leaf node aggregation layer are used to represent the model names of different candidate generation models; and nodes in different non-leaf node aggregation layers are used to represent categories under different attribute dimensions.

[0088] In addition, the present application does not limit the construction process of the above model tree. For example, in some application scenarios, the model tree can be manually constructed by relevant personnel.

[0089] For another example, in some application scenarios, the above model tree can be automatically constructed based on the relevant information (such as model name, model attributes, etc.) of at least two candidate generation models above. It can be seen that, in a possible implementation, the model tree can be constructed based on the model description information of at least two candidate generation models above. Among them, the model description information of the i-th candidate generation model is used to describe the characteristics of the i-th candidate generation model; and the present application does not limit the implementation of the model description information of the i-th candidate generation model. For example, in a possible implementation, the model description information of the i-th candidate generation model may include the model name of the i-th candidate generation model and at least one model attribute of the i-th candidate generation model. Among them, the model name of the i-th candidate generation model is used to uniquely identify the i-th candidate generation model; and the present application does not limit the model name of the i-th candidate generation model. At least one model attribute of the i-th candidate generation model is used to indicate the attribute state of the candidate generation model; and the present application does not limit at least one model attribute of the i-th candidate generation model. i is a positive integer, i≤I, I is a positive integer, and I represents the number of models in the at least two candidate generation models.

[0090] In addition, the present application does not limit the automatic construction process of the model tree in the above paragraph. For example, in one possible implementation, the construction process of the model tree may specifically include the following steps 11 and 12.

[0091] Step 11: Obtain at least one attribute dimension.

[0092] It should be noted that for the relevant content of step 11, please refer to the relevant content of "at least one attribute dimension" above. For the sake of brevity, it will not be repeated here.

[0093] Step 12: Based on the model names of the at least two candidate generation models above and the classification results of the at least two candidate generation models under each attribute dimension, construct a model tree so that different nodes in the leaf node aggregation layer of the model tree are used to represent the model names of different candidate generation models, and the nodes in different non-leaf node aggregation layers of the model tree are used to represent categories under different attribute dimensions.

[0094] In the present application, after obtaining the classification results of the at least two candidate generation models under the jth attribute dimension, the jth non-leaf node aggregation layer in the above model tree can be constructed based on the classification results, so that different nodes in the jth non-leaf node aggregation layer can represent different categories under the jth attribute dimension, so that the jth non-leaf node aggregation layer can represent the classification status of the at least two candidate generation models under the jth attribute dimension, j is a positive integer, j≤J, J is a positive integer, so that after all non-leaf node aggregation layers in the model tree are constructed, the nodes used to represent the model names of each candidate generation model are added to the model tree as leaf nodes, so that different nodes in the leaf node aggregation layer of the finally constructed model tree are used to represent the model names of different candidate generation models, and the nodes in different non-leaf node aggregation layers of the model tree are used to represent categories under different attribute dimensions, so that the finally constructed model tree can represent the category attribution status of the at least two candidate generation models.

[0095] Based on the relevant contents of steps 11 to 12 above, it can be known that in some application scenarios, when the above model tree may include at least one non-leaf node aggregation layer (for example, Figure 3 The categories represented by the nodes in different non-leaf node aggregation layers are determined based on the classification results of at least two candidate generation models under different attribute dimensions, so that the model tree is used to describe the category affiliation status of the at least two candidate generation models (for example, information such as the categories they belong to under different attribute dimensions).

[0096] Based on the relevant content of the model tree above, it can be known that for the model tree provided in the present application, the model tree can be used to describe the category affiliation status of at least two candidate generation models above, so that the model tree can represent the classification results of these candidate generation models under different attribute dimensions, thereby enabling the model tree to better represent the characteristics of these candidate generation models, which is conducive to improving the execution efficiency of selecting a model suitable for image generation processing for the target text from these candidate generation models.

[0097] The above “target generation model corresponding to the target text” refers to a model selected from the above at least two candidate generation models with the help of the above model tree, which is suitable for image generation processing for the target text, so that the target generation model can be used to perform image generation processing for the target text later. For example, when the model tree is Figure 2 In the idea tree shown, the “target generation model corresponding to the target text” may refer to model 1.

[0098] In addition, the present application does not limit the determination process of the above “target generation model corresponding to the target text”. For ease of understanding, some possible implementation methods are described below.

[0099] In a possible implementation manner, the determination process of the above “target generation model corresponding to the target text” may specifically include the following steps 21 and 22.

[0100] Step 21: Search the model tree for non-leaf nodes that match the target text to obtain a matching result, so that the matching result can represent the category affiliation status of the image generation model suitable for performing image generation processing on the target text.

[0101] The matching result is used to describe the non-leaf nodes in the model tree above that match the target text, so that the matching result can represent the category attribution status of the image generation model suitable for image generation processing for the target text. For example, when the model tree is Figure 3 In the thought tree shown, the matching result may include node 3 and node 6, so that the matching result can indicate that the image generation model suitable for image generation processing for the target text belongs to the category represented by node 3 (for example, the subject category of "animal") and the category represented by node 6 (for example, the style category of "ink painting").

[0102] In addition, the present application does not limit the process of determining the above matching results. For example, it can be implemented using any existing or future method that can search and process a tree structure (for example, a breadth-first search method, etc.).

[0103] In fact, in some application scenarios, there may be some content with less use value in the above target text. Therefore, in order to improve the matching efficiency, the present application also provides a possible implementation method of the process of determining the above matching results. Under this implementation method, the process of determining the matching results can specifically include the following steps 211-212.

[0104] Step 211: extract the core content of the target text to obtain the core content.

[0105] The core content is used to represent the core content described in the target text above. For example, when the target text is Figure 2 When the text “generate an image of a dog” is shown, the core content may be the content “an image of a dog”.

[0106] In addition, the present application does not limit the implementation method of step 211. For example, it can be implemented by any existing or future method that can extract the core content of a text content.

[0107] In addition, in order to better improve the accuracy of core content extraction, the present application also provides a possible implementation of the above step 211. Under this implementation, the step 211 may specifically include the following steps 2111-2112.

[0108] Step 2111: Determine a target style that matches the above target text from the extracted text styles corresponding to at least two core content extraction algorithms.

[0109] The core content extraction algorithm is used to perform core content extraction processing on a text content.

[0110] In addition, for at least two core content extraction algorithms above (e.g., Figure 2For the extraction algorithms 1 to N shown in the figure, N is a positive integer), different core content extraction algorithms have different working principles. For example, in some application scenarios, the at least two core content extraction algorithms may include a prompt core content extraction algorithm, an instruction core content extraction algorithm, an inspiration core content extraction algorithm, and a hypothesis core content extraction algorithm. Among them, the prompt core content extraction algorithm is more suitable for performing core content extraction processing on the text content expressed in the prompt style (for example, the text content of "a dog"), and the working principle of the prompt core content extraction algorithm can be specifically: directly treating the entire text as the core content (for example, the core content of "a dog"). The instruction core content extraction algorithm is more suitable for performing core content extraction processing on the text content expressed in the instruction style (for example, the text content of "generating an image of a dog"), and the working principle of the instruction core content extraction algorithm can be specifically: extracting the core part behind the instruction as the core content (for example, the core content of "an image of a dog"). The inspiration core content extraction algorithm is more suitable for extracting core content from text content expressed in an inspiration style (e.g., the text content "I want to go to the beach"), and the working principle of the inspiration core content extraction algorithm can be specifically: extracting the target subject of desire as the core content (e.g., the core content of "beach"). The hypothesis core content extraction algorithm is more suitable for extracting core content from text content expressed in a hypothesis style (e.g., the text content "If you give me a bouquet of flowers, I will smile happily"), and the working principle of the hypothesis core content extraction algorithm can be specifically: extracting the hypothesis conditions and the object of the action to be collected as the core content (e.g., the core content of "a bouquet of flowers and a smiling face").

[0111] For at least two of the above core content extraction algorithms (e.g., Figure 2 For the extraction algorithms 1 to N shown in the figure, where N is a positive integer), different core content extraction algorithms are suitable for performing core content extraction processing on texts of different styles, so that the at least two core content extraction algorithms can be suitable for performing core content extraction processing on texts of multiple styles (for example, text content expressed in a prompt style, text content expressed in an instruction style, text content expressed in an inspiration style, or text content expressed in a hypothesis style, etc.), which is conducive to improving the accuracy of core content extraction.

[0112] In addition, for the nth core content extraction algorithm, if the nth core content extraction algorithm is suitable for performing core content extraction processing on the text content of the nth style, then the extracted text style corresponding to the nth core content extraction algorithm is the nth style (for example, a prompt style, an instruction style, an inspiration style, or a hypothesis style, etc.). It should be noted that the present application does not limit the implementation method of the nth style above, for example, it can be set in advance according to the actual application scenario. Wherein, n is a positive integer, n≤N, N is a positive integer, and N represents the number of algorithms in the at least two core content extraction algorithms above.

[0113] The target style refers to a style that exists in the extracted text styles corresponding to at least two core content extraction algorithms above and matches the target text, so that the target style can represent the text style that the target text conforms to.

[0114] In addition, the present application does not limit the determination process of the above target style. For example, it can be specifically as follows: after obtaining the target text, determine whether the target text conforms to the extraction text style corresponding to the nth core content extraction algorithm. If it conforms, then the extraction text style corresponding to the nth core content extraction algorithm is determined as the target style, so that the target text can be subsequently processed by using the nth core content extraction algorithm corresponding to the target style. Wherein, n is a positive integer, n≤N, N is a positive integer, and N represents the number of algorithms in the at least two core content extraction algorithms above.

[0115] Based on the relevant content of step 2111 above, it can be known that after obtaining the target text, a target style that matches the above target text can be determined from the extracted text styles corresponding to some pre-specified core content extraction algorithms with better core content extraction performance, so that the core content extraction algorithm corresponding to the target style can be used to perform core content extraction processing on the target text.

[0116] Step 2112: Using the core content extraction algorithm corresponding to the above target style, perform core content extraction processing on the above target text to obtain the core content.

[0117] The core content extraction algorithm corresponding to the target style is used to represent a method suitable for performing core content extraction processing on the text content expressed according to the target style. For example, if the target style is a prompt style, the core content extraction algorithm corresponding to the target style may be a prompt core content extraction algorithm; if the target style is an instruction style, the core content extraction algorithm corresponding to the target style may be an instruction core content extraction algorithm; if the target style is an inspiration style, the core content extraction algorithm corresponding to the target style may be an inspiration core content extraction algorithm; if the target style is a hypothesis style, the core content extraction algorithm corresponding to the target style may be a hypothesis core content extraction algorithm.

[0118] Based on the relevant contents of steps 2111 to 2112 above, it can be known that in some application scenarios, after obtaining the text content input by the user (for example, the target text above), it can be first determined that the text style presented by the text content is closer to which of the at least two core content extraction algorithms above the corresponding extracted text style of the core content extraction algorithm, so that after determining that the text style presented by the text content is closest to the extracted text style corresponding to the target core content extraction algorithm, the target core content extraction algorithm is directly used to perform core content extraction processing on the text content to obtain the core content, so that the core content can represent the core information carried by the text content, which is conducive to eliminating the impact of noise text on the image generation process, thereby facilitating the generation of high-quality images. Among them, the target core content extraction algorithm refers to the algorithm that is most suitable for performing core content extraction processing on the text content among the at least two core content extraction algorithms.

[0119] Based on the relevant content of step 211 above, it can be known that in some application scenarios, after obtaining the target text input by the user, the target text can be subjected to core content extraction processing to obtain the core content, so that the core content can represent the core information carried by the target text, which is conducive to eliminating the impact of noise text on the image generation process, thereby facilitating the generation of high-quality images.

[0120] Step 212: Search the model tree for non-leaf nodes that match the core content above to obtain matching results.

[0121] It should be noted that the present application does not limit the implementation method of the above step 212. For example, it can be specifically: according to the preset search method, search for non-leaf nodes matching the above core content from the model tree to obtain matching results, so that the matching results can indicate which non-leaf nodes in the model tree match the core content, so that the matching results can indicate the category belonging state of the model suitable for image processing of the above target text. Among them, the preset search algorithm is used to search for certain content from the model tree; and the present application does not limit the implementation method of the preset search algorithm. For example, it can be implemented by using a breadth-first search algorithm.

[0122] Based on the content of the previous paragraph, it can be known that in a possible implementation mode, the above step 212 can specifically be: searching for nodes that match the above core content from each non-leaf node aggregation layer of the model tree in turn, and obtaining matching results, so that the matching results can represent the nodes that match the core content obtained by searching from each non-leaf node aggregation layer.

[0123] It should be noted that the present application does not limit the implementation method of the step of "searching for nodes matching the core content of the above text from each non-leaf node aggregation layer of the model tree in sequence" in the above paragraph. For example, it can be specifically: searching for nodes matching the core content of the above text from the first non-leaf node aggregation layer of the model tree, and obtaining the matching nodes corresponding to the first non-leaf node aggregation layer, so that the matching nodes corresponding to the first non-leaf node aggregation layer can represent the category that best matches the core content under the first attribute dimension; searching for nodes matching the core content from all child nodes of the matching nodes corresponding to the first non-leaf node aggregation layer (that is, the nodes in the second non-leaf node aggregation layer that have the matching nodes corresponding to the first non-leaf node aggregation layer as their parent nodes), and obtaining the matching nodes corresponding to the second non-leaf node aggregation layer, so that the matching nodes corresponding to the second non-leaf node aggregation layer can represent the category that best matches the core content under the second attribute dimension. Differentiation; searching for nodes matching the core content from all child nodes of the matching node corresponding to the second non-leaf node aggregation layer (that is, the nodes in the third non-leaf node aggregation layer that have the matching node corresponding to the second non-leaf node aggregation layer as their parent node) to obtain the matching nodes corresponding to the third non-leaf node aggregation layer, so that the matching nodes corresponding to the third non-leaf node aggregation layer can represent the category that best matches the core content under the third attribute dimension; ... (and so on), so that the matching result can be determined later based on the matching nodes corresponding to all non-leaf node aggregation layers (for example, the matching nodes corresponding to the first non-leaf node aggregation layer, the matching nodes corresponding to the second non-leaf node aggregation layer, ...), so that the matching result includes the matching nodes corresponding to all non-leaf node aggregation layers, so that the matching result can represent the category attribution status of the model suitable for image processing of the target text above under each attribute dimension.

[0124] Based on the relevant content of steps 211 to 212 above, it can be known that in some application scenarios, after obtaining the target text input by the user, the target text can be first subjected to core content extraction processing to obtain the core content, so that the core content can represent the core information carried by the target text; then search the model tree for non-leaf nodes that match the core content to obtain a matching result, so that the matching result can better represent the category affiliation status of the image generation model suitable for performing image generation processing on the target text.

[0125] In fact, in some application scenarios, in order to better improve the matching efficiency, the present application also provides another possible implementation of the above step 21. In this implementation, the step 21 may specifically include the following steps 213 to 215.

[0126] Step 213: Perform core content extraction processing on the target text to obtain the core content.

[0127] It should be noted that the relevant contents of step 213 above can be referred to the relevant contents of step 211 above, and for the sake of brevity, they will not be repeated here.

[0128] Step 214: extract key content from the core content of the above text to obtain at least one key content.

[0129] Among them, the key content extraction process is used to extract some content with higher value (for example, keywords, key phrases, etc.) from a text content; and the present application does not limit the implementation method of the key content extraction process. For example, it can be implemented using any existing or future method that can extract content with higher value from a text content (for example, keyword extraction method, etc.).

[0130] It should be noted that the present application does not limit the implementation method of the key content. For example, it can be implemented using keywords, key words, key sentences, etc.

[0131] In addition, for the at least one key content above, the at least one key content is used to represent information with higher value present in the core content above; and the present application does not limit the implementation method of the at least one key content, for example, it may include one or more of at least one keyword, at least one keyword and at least one key sentence present in the core content.

[0132] Step 215: Search the model tree for a non-leaf node that matches at least one key content in the above text to obtain a matching result.

[0133] It should be noted that the present application does not limit the implementation method of the above step 215. For example, it can be specifically: according to the preset search method, search for non-leaf nodes matching at least one key content in the above text from the model tree to obtain a matching result, so that the matching result can indicate which non-leaf nodes in the model tree match the at least one key content, so that the matching result can indicate the category belonging state of the model suitable for image processing of the target text in the above text. Among them, the preset search algorithm is used to search for certain content from the model tree; and the present application does not limit the implementation method of the preset search algorithm. For example, it can be implemented by using a breadth-first search algorithm.

[0134] Based on the content of the previous paragraph, it can be known that in a possible implementation mode, the above step 215 can specifically be: searching for nodes that match at least one key content above from each non-leaf node aggregation layer of the model tree in turn, and obtaining a matching result, so that the matching result can represent the nodes that match the at least one key content obtained by searching from each non-leaf node aggregation layer.

[0135] It should be noted that the present application does not limit the implementation method of the step of "searching for nodes matching at least one key content above from each non-leaf node aggregation layer of the model tree in turn" in the above paragraph. For example, it can be specifically: searching for nodes matching at least one key content above from the first non-leaf node aggregation layer of the model tree, and obtaining matching nodes corresponding to the first non-leaf node aggregation layer, so that the matching nodes corresponding to the first non-leaf node aggregation layer can represent the category that best matches the at least one key content under the first attribute dimension; searching for nodes matching the at least one key content from all child nodes of the matching node corresponding to the first non-leaf node aggregation layer (that is, the nodes in the second non-leaf node aggregation layer that have the matching node corresponding to the first non-leaf node aggregation layer as the parent node), and obtaining matching nodes corresponding to the second non-leaf node aggregation layer, so that the matching nodes corresponding to the second non-leaf node aggregation layer can represent the category that best matches the at least one key content under the second attribute dimension. the category that best matches the content; searching for nodes that match the at least one key content from all child nodes of the matching node corresponding to the second non-leaf node aggregation layer (that is, the nodes in the third non-leaf node aggregation layer that have the matching node corresponding to the second non-leaf node aggregation layer as their parent node) to obtain the matching nodes corresponding to the third non-leaf node aggregation layer, so that the matching nodes corresponding to the third non-leaf node aggregation layer can represent the category that best matches the at least one key content under the third attribute dimension; ... (and so on), so that the matching result can be determined later based on the matching nodes corresponding to all non-leaf node aggregation layers (for example, the matching nodes corresponding to the first non-leaf node aggregation layer, the matching nodes corresponding to the second non-leaf node aggregation layer, ...), so that the matching result includes the matching nodes corresponding to all non-leaf node aggregation layers, so that the matching result can represent the category attribution status of the model suitable for image processing of the target text above under each attribute dimension.

[0136] Based on the relevant content of steps 213 to 215 above, it can be known that in some application scenarios, after obtaining the target text input by the user, the target text can be first subjected to core content extraction processing to obtain the core content, so that the core content can represent the core information carried by the target text; then at least one keyword content is extracted from the core content, so that the at least one keyword content can represent the core information carried by the target text as briefly as possible; then, a non-leaf node matching the at least one keyword content is searched from the model tree to obtain a matching result, so that the matching result can better represent the category affiliation status of the image generation model suitable for performing image generation processing on the target text.

[0137] In fact, in some application scenarios, in order to better improve the matching efficiency, the present application also provides another possible implementation of the above step 21. In this implementation, when the above model tree includes J non-leaf node aggregation layers arranged in sequence (for example, Figure 3 The 2nd to 3rd layers in the idea tree shown), each non-leaf node aggregation layer includes at least two non-leaf nodes, and when J is a positive integer, the step 21 can specifically include the following steps 216-218.

[0138] Step 216: Search the first non-leaf node aggregation layer for a node that matches the target text, and obtain the target node corresponding to the first non-leaf node aggregation layer.

[0139] The first non-leaf node aggregation layer refers to the non-leaf node aggregation layer that is arranged at the front in the model tree above. For example, when the model tree is Figure 3 When the idea tree is shown, the first non-leaf node aggregation layer can be Figure 3 Level 2 in the idea tree shown.

[0140] In addition, for the first non-leaf node aggregation layer in the above model tree (for example, Figure 3 For example, the first non-leaf node aggregation layer includes at least two non-leaf nodes (for example, Figure 3 Node 2 and Node 3 shown); and the parent node of each node in the first non-leaf node aggregation layer is the root node of the model tree (for example, Figure 3 Node 1 shown).

[0141] In addition, for the first non-leaf node aggregation layer in the above model tree, the target node corresponding to the first non-leaf node aggregation layer refers to the node in the first non-leaf node aggregation layer that matches the semantic information carried by the target text, so that the target node corresponding to the first non-leaf node aggregation layer can represent the category that matches the target text under the first attribute dimension. For example, when the first non-leaf node aggregation layer is Figure 3 When the second layer in the thought tree shown in FIG. 1 is a layer 2, the target node corresponding to the first non-leaf node aggregation layer may refer to Figure 3 Node 3 is shown.

[0142] In addition, the present application does not limit the implementation method of the above step 216. For example, in some application scenarios, the step 216 can specifically be: after obtaining the target text, directly search for the node matching the target text from the first non-leaf node aggregation layer of the model tree to obtain the target node corresponding to the first non-leaf node aggregation layer.

[0143] For example, in some application scenarios, the above step 216 can be specifically as follows: after obtaining the target text, first perform core content extraction processing on the target text to obtain the core content; then search for nodes matching the core content from the first non-leaf node aggregation layer of the model tree to obtain the target node corresponding to the first non-leaf node aggregation layer.

[0144] For example, in some application scenarios, the above step 216 can be specifically as follows: after acquiring the target text, first perform core content extraction processing on the target text to obtain the core content; then perform key content extraction processing on the core content to obtain at least one key content; then, search for nodes corresponding to the at least one key content from the first non-leaf node aggregation layer of the model tree to obtain the target node corresponding to the first non-leaf node aggregation layer.

[0145] Based on the relevant content of step 216 above, it can be known that in some application scenarios, after obtaining the target text, the node matching the target text can be searched from the first non-leaf node aggregation layer of the model tree to obtain the target node corresponding to the first non-leaf node aggregation layer (for example, Figure 3 Node 3) shown in the figure, so that the matching result corresponding to the target text can be determined based on the target node later.

[0146] Step 217: Search for nodes matching the target text from the search range corresponding to the j-th non-leaf node aggregation layer to obtain the target node corresponding to the j-th non-leaf node aggregation layer; the search range corresponding to the j-th non-leaf node aggregation layer is determined based on the target node corresponding to the j-1-th non-leaf node aggregation layer; j is a positive integer, 2≤j≤J.

[0147] The j-th non-leaf node aggregation layer refers to the non-leaf node aggregation layer at the j-th arrangement position in the above model tree, and j is a positive integer, 2≤j≤J.

[0148] In addition, for the jth non-leaf node aggregation layer in the above model tree (for example, Figure 3 For example, the jth non-leaf node aggregation layer includes at least two non-leaf nodes (for example, Figure 3 Nodes 4 to 7 shown); and the parent nodes of each node in the j-th non-leaf node aggregation layer are all located in the j-1-th non-leaf node aggregation layer in the model tree (for example, Figure 3 The j-1st non-leaf node aggregation layer refers to the non-leaf node aggregation layer at the j-1st arrangement position in the above model tree, j is a positive integer, 2≤j≤J.

[0149] Based on the content of the previous paragraph, it can be known that for the above model tree, the parent node of each node in the second non-leaf node aggregation layer in the model tree is located in the first non-leaf node aggregation layer; the parent node of each node in the third non-leaf node aggregation layer in the model tree is located in the second non-leaf node aggregation layer; ... (and so on); the parent node of each node in the Jth non-leaf node aggregation layer in the model tree is located in the J-1th non-leaf node aggregation layer.

[0150] In addition, for the j-th non-leaf node aggregation layer in the above model tree, the search range corresponding to the j-th non-leaf node aggregation layer is determined based on the target node corresponding to the j-1-th non-leaf node aggregation layer, so that the parent node of each node in the search range corresponding to the j-th non-leaf node aggregation layer is the target node corresponding to the j-1-th non-leaf node aggregation layer. Among them, the search range corresponding to the j-th non-leaf node aggregation layer refers to the search range required to be based on when searching for the j-th non-leaf node aggregation layer when the search results for the j-1-th non-leaf node aggregation layer are determined; and the search range corresponding to the j-th non-leaf node aggregation layer may include at least one node existing in the j-th non-leaf node aggregation layer, which has the target node corresponding to the j-1-th non-leaf node aggregation layer as a parent node, j is a positive integer, 2≤j≤J.

[0151] Based on the above content, it can be known that in a possible implementation, the search range corresponding to the jth non-leaf node aggregation layer above may include at least one child node of the target node corresponding to the j-1th non-leaf node aggregation layer. For example, when the jth non-leaf node aggregation layer is Figure 3The third layer in the thought tree shown, and the corresponding target node of the j-1th non-leaf node aggregation layer is Figure 3 When the node 3 is shown, the search range corresponding to the jth non-leaf node aggregation layer may include Figure 3 Node 6 and node 7 are shown. Wherein, j is a positive integer, 2≤j≤J.

[0152] In addition, for the jth non-leaf node aggregation layer in the above model tree, the target node corresponding to the jth non-leaf node aggregation layer refers to the node that matches the semantic information carried by the target text in the search range corresponding to the jth non-leaf node aggregation layer, so that the target node corresponding to the jth non-leaf node aggregation layer can represent the category that matches the target text under the jth attribute dimension, j is a positive integer, 2≤j≤J. For example, when the jth non-leaf node aggregation layer is Figure 3 The second layer in the thought tree shown, and the search range corresponding to the j-th non-leaf node aggregation layer includes Figure 3 When the node 6 and the node 7 are shown, the target node corresponding to the jth non-leaf node aggregation layer can be Figure 3 Node 6 is shown.

[0153] In addition, the present application does not limit the implementation method of the above step 217. For example, it can be specifically as follows: after determining the target node corresponding to the j-1th non-leaf node aggregation layer in the above model tree, first determine the search range corresponding to the j-1th non-leaf node aggregation layer based on the target node corresponding to the j-1th non-leaf node aggregation layer, so that the search range includes at least one child node of the target node corresponding to the j-1th non-leaf node aggregation layer; then directly search for nodes matching the target text from the search range corresponding to the j-1th non-leaf node aggregation layer to obtain the target node corresponding to the j-1th non-leaf node aggregation layer. Wherein, j is a positive integer, 2≤j≤J.

[0154] For another example, in some application scenarios, the above step 217 can be specifically as follows: after determining the target node corresponding to the j-1th non-leaf node aggregation layer in the above model tree, first determine the search range corresponding to the j-1th non-leaf node aggregation layer based on the target node corresponding to the j-1th non-leaf node aggregation layer, so that the search range includes at least one child node of the target node corresponding to the j-1th non-leaf node aggregation layer; then directly search for nodes matching the core content of the target text from the search range corresponding to the j-1th non-leaf node aggregation layer to obtain the target node corresponding to the j-1th non-leaf node aggregation layer. Wherein, j is a positive integer, 2≤j≤J. It should be noted that the relevant content of the core content can be found above.

[0155] For example, in some application scenarios, the above step 217 can be specifically as follows: after determining the target node corresponding to the j-1th non-leaf node aggregation layer in the above model tree, first determine the search range corresponding to the j-1th non-leaf node aggregation layer based on the target node corresponding to the j-1th non-leaf node aggregation layer, so that the search range includes at least one child node of the target node corresponding to the j-1th non-leaf node aggregation layer; then directly search for nodes matching at least one key content of the target text from the search range corresponding to the j-1th non-leaf node aggregation layer to obtain the target node corresponding to the j-1th non-leaf node aggregation layer. Wherein, j is a positive integer, 2≤j≤J. It should be noted that the relevant content of the at least one key content can be found above.

[0156] Based on the relevant content of step 217 above, it can be known that for the above model tree, after determining the target node corresponding to the j-1th non-leaf node aggregation layer in the model tree, the node matching the target text can be searched from all child nodes of the target node corresponding to the j-1th non-leaf node aggregation layer (that is, the search range corresponding to the jth non-leaf node aggregation layer) to obtain the target node corresponding to the jth non-leaf node aggregation layer, so that the matching result corresponding to the target text can be determined based on the target node later, where j is a positive integer, 2≤j≤J, which is conducive to improving search efficiency.

[0157] Step 218: Determine the matching result based on the target nodes corresponding to the J non-leaf node aggregation layers.

[0158] In the present application, for the above model tree, after obtaining the target nodes corresponding to each non-leaf node aggregation layer in the model tree, the matching results corresponding to the above target text can be determined directly based on these target nodes, so that the matching results include the target nodes corresponding to all non-leaf node aggregation layers in the model tree, so that the matching results can represent the categories represented by these target nodes, and further enable the matching results to represent the category affiliation status of the image generation model suitable for image generation processing for the target text.

[0159] Based on the relevant contents of steps 216 to 218 above, it can be known that in some application scenarios, after obtaining the target text input by the user, the category matching the target text can be searched layer by layer in at least two non-leaf node aggregation layers of the model tree to obtain a matching result, so that the matching result can represent the category belonging state of the image generation model suitable for image generation processing for the target text. Among them, for any other non-leaf node aggregation layer in the model tree except the non-leaf node aggregation layer with the most forward arrangement position, the search range required for searching for the target text in the non-leaf node aggregation layer only includes some nodes in the non-leaf node aggregation layer, which is conducive to improving the search efficiency for the non-leaf node aggregation layer, thereby helping to improve the execution efficiency of matching the target text in the model tree.

[0160] Based on the relevant content of step 21 above, it can be known that in some application scenarios, after obtaining the target text input by the user, the categories matching the semantic information carried by the target text can be searched layer by layer from the pre-constructed model tree to obtain matching results, so that the matching results can represent the category affiliation status of the image generation model suitable for performing image generation processing on the target text, so that the image generation model suitable for performing image generation processing on the target text can be determined based on the matching results.

[0161] Step 22: Based on the above matching results, determine the target generation model corresponding to the target text.

[0162] It should be noted that the present application does not limit the implementation method of the above step 22. For example, it can be specifically as follows: after obtaining the matching result corresponding to the above target text, first search the leaf node corresponding to the matching result from the above model tree, so that the candidate generation model represented by the leaf node has the category belonging state described by the matching result; then determine the target generation model corresponding to the target text from the candidate generation model represented by the leaf node (for example, if there is only one leaf node corresponding to the matching result, then directly determine the candidate generation model represented by the leaf node as the target generation model corresponding to the target text; if there are multiple leaf nodes corresponding to the matching result, then select a model from the candidate generation models represented by the multiple leaf nodes according to a preset method as the target generation model corresponding to the target text). Among them, the preset method can be pre-set, for example, the preset method can be a random selection. For example, the preset method can be a selection according to a certain pre-set selection rule, etc.

[0163] In fact, in order to better improve the accuracy of model selection, the present application also provides a possible implementation of the above step 22. Under this implementation, the step 22 may specifically include the following steps 221-222.

[0164] Step 221: After obtaining the matching result corresponding to the above target text, determine at least one candidate leaf node corresponding to the matching result from the model tree.

[0165] The candidate leaf node refers to the leaf node existing in the model tree above, which has all the nodes in the matching result corresponding to the target text above as direct nodes, so that the candidate generation model represented by the candidate leaf node has the category belonging state described by the matching result. That is, for the yth candidate leaf node (for example, Figure 3 For example, for node 12 or node 13 shown in FIG. 1 , the matching result corresponding to the target text may include the parent node of the yth candidate leaf node (for example, Figure 3 Node 6 shown), the grandparent node of the y-th candidate leaf node (for example, Figure 3 Node 3 shown), ... (and so on). y is a positive integer, y≤Y, Y is a positive integer, and Y represents the number of leaf nodes in the above "at least one candidate leaf node".

[0166] Based on the relevant content of step 221 above, it can be known that for some application scenarios, after searching the model tree for the matching result corresponding to the target text (for example, Figure 3 After nodes 3 and 6 as shown), the model tree can be searched for leaf nodes with all the nodes in the matching result as direct nodes, as candidate leaf nodes corresponding to the matching result, so that the image generation model that is most suitable for image generation processing for the target text can be selected from the candidate generation models represented by these candidate leaf nodes.

[0167] Step 222: Determine the target generation model corresponding to the target text based on the data representing the degree of match between the target text and the model description information corresponding to each candidate leaf node.

[0168] Among them, the model description information corresponding to the yth candidate leaf node is used to describe the characteristics of the candidate generation model represented by the yth candidate leaf node; and the present application does not limit the model description information of the yth candidate leaf node. For example, in some application scenarios, the model description information of the yth candidate leaf node may include the model name corresponding to the yth candidate leaf node and at least one model attribute corresponding to the yth candidate leaf node. Among them, the model name corresponding to the yth candidate leaf node is used to uniquely identify the candidate generation model represented by the yth candidate leaf node. The at least one model attribute corresponding to the yth candidate leaf node is used to describe the attribute state of the candidate generation model represented by the yth candidate leaf node; and the present application does not limit the at least one model attribute corresponding to the yth candidate leaf node. For example, in some application scenarios, the at least one model attribute corresponding to the yth candidate leaf node may include one or more attribute tags pre-annotated for the candidate generation model represented by the yth candidate leaf node. y is a positive integer, y≤Y, Y is a positive integer, and Y represents the number of leaf nodes in the above "at least one candidate leaf node".

[0169] In addition, for the yth candidate leaf node above, the matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node above is used to describe the matching degree between the target text and the model description information corresponding to the yth candidate leaf node, so that the "matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node" can represent the degree of adaptation between the target text and the candidate generation model represented by the yth candidate leaf node, so that the "matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node" can represent the possibility of using the candidate generation model represented by the yth candidate leaf node to perform image generation processing on the target text. y is a positive integer, y≤Y, Y is a positive integer, and Y represents the number of leaf nodes in the above "at least one candidate leaf node".

[0170] In addition, the present application does not limit the method for determining the above-mentioned "data representing the degree of match between the target text and the model description information corresponding to the yth candidate leaf node". For example, it can be implemented by any existing or future method that can measure the degree of match between the two pieces of information (for example, by processing with the help of a pre-built language model).

[0171] For another example, in some application scenarios, the determination process of the above "matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node" includes: calculating the matching degree representation data between the core content of the target text and the model description information corresponding to the yth candidate leaf node as the matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node. It should be noted that the relevant content of the core content can be found above.

[0172] For example, the determination process of the above "matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node" includes: calculating the matching degree representation data between at least one key content of the target text and the model description information corresponding to the yth candidate leaf node as the matching degree representation data between the target text and the model description information corresponding to the yth candidate leaf node. It should be noted that the relevant content of the at least one key content can be found above.

[0173] In addition, the present application does not limit the implementation method of the above step 222. For example, it can be specifically as follows: after determining at least one candidate leaf node corresponding to the above target text from the model tree, based on the data representing the degree of matching between the target text and the model description information corresponding to each candidate leaf node, a model that satisfies the preset matching condition is determined from the candidate generation models represented by the at least one candidate leaf node as the target generation model corresponding to the target text. Among them, the preset matching condition refers to the condition satisfied by the candidate generation model finally selected for a text content (for example, the above target text); and the present application does not limit the implementation method of the preset matching condition. For example, the preset matching condition can be specifically as follows: the degree of matching between the candidate leaf node representing the target generation model corresponding to the target text and the target text reaches the highest (that is, the highest degree of adaptation).

[0174] Based on the relevant content of step 222 above, it can be known that for some application scenarios, after searching for at least one candidate leaf node for the target text from the model tree, the degree of adaptation between the target text and the candidate generation model represented by each candidate leaf node can be determined based on the model description information of each candidate leaf node; and then the model with the highest degree of adaptation is selected from the candidate generation models represented by the at least one candidate leaf node as the target generation model corresponding to the target text, so that the target generation model can represent the image generation model that is most suitable for image generation processing for the target text.

[0175] Based on the relevant contents of steps 221 to 222 above, it can be known that for some application scenarios, after searching the model tree for the matching result corresponding to the target text (for example, Figure 3After nodes 3 and 6 as shown, at least one candidate leaf node corresponding to the target text can be determined from the model tree based on the matching result; then, based on the target text and the model description information of the candidate generation models represented by these candidate leaf nodes, the degree of adaptation between the target text and the candidate generation models represented by each candidate leaf node can be determined; and then, the model with the highest degree of adaptation is selected from the candidate generation models represented by the at least one candidate leaf node as the target generation model corresponding to the target text, so that the target generation model can represent the image generation model that is most suitable for image generation processing for the target text.

[0176] Based on the relevant content of S2 above, it can be known that in some application scenarios, after obtaining the target text input by the user, it is possible to obtain the target text from the pre-built model tree (for example, Figure 3 The idea tree shown in FIG. 1 is used to search for a candidate model set that matches the target text (for example, Figure 3 The image generation model represented by each node in the candidate set shown in the figure); then, based on the core content of the target text (for example, the entire text content, the core content or at least one key content, etc.) and the model description information of the candidate generation models represented by these candidate leaf nodes, the image generation model that is most suitable for image generation processing for the target text is selected from the candidate model set as the target generation model corresponding to the target text.

[0177] S3: Generate image data corresponding to the target text using the target generation model.

[0178] In the present application, after obtaining the target generation model corresponding to the target text above, the target generation model can be directly used to generate image data corresponding to the target text (for example, directly input the target text into the target generation model so that the target generation model performs image generation processing on the target text, obtains and outputs the image data corresponding to the target text). Among them, because the target generation model is more suitable for performing image generation processing on the target text, the image data generated by the target generation model for the target text can be more consistent with the semantic information expressed by the target text, which is conducive to improving the image generation effect.

[0179] In fact, in order to better improve the image generation effect, the present application also provides a possible implementation of the above S3. Under this implementation, the S3 may specifically include the following steps 31 and 32.

[0180] Step 31: After obtaining the target generation model corresponding to the above target text, the target text is updated according to the text format constraint information corresponding to the target generation model to obtain an updated text.

[0181] The text format constraint information corresponding to the target generation model is used to indicate the text format for which the target generation model is more suitable for image generation processing.

[0182] In addition, the present application does not limit the implementation method of the above "text format constraint information corresponding to the target generation model". For example, in some application scenarios, the "text format constraint information corresponding to the target generation model" may include constraint description content under one or more format dimensions. It should be noted that the present application does not limit the implementation method of the one or more format dimensions. For example, it can be set according to the actual application scenario. In addition, the present application does not limit the implementation method of the constraint description content of the one or more format dimensions. For example, it can be set according to the actual application scenario.

[0183] For another example, in some application scenarios, the above “text format constraint information corresponding to the target generation model” may include a processed text sample corresponding to the target generation model. The processed text sample refers to a text example pre-set for the target generation model and suitable for image processing using the target generation model, so that the processed text sample can indicate what format of text the target generation model is more suitable for image generation processing.

[0184] In addition, the present application does not limit the number of the above processed text samples. For example, in some application scenarios, the above "text format constraint information corresponding to the target generation model" may include one or more processed text samples corresponding to the target generation model, so that the "text format constraint information corresponding to the target generation model" can indicate as comprehensively and accurately as possible what format of text the target generation model is more suitable for image generation processing.

[0185] In addition, the present application does not limit the method of obtaining the above processed text sample, for example, the processed text sample can be preset. For another example, the processed text sample can be automatically determined based on the training data of the above target generation model.

[0186] The updated text refers to the text obtained by updating the target text according to the text format constraint information corresponding to the target generation model above, so that the semantic information expressed by the updated text is consistent with the semantic information expressed by the target text, and the updated text satisfies the text format described by the text format constraint information, so that the updated text can express the semantic information expressed by the target text on the premise of satisfying the text format described by the text format constraint information, and further enables the updated text to express the semantic information expressed by the target text according to the text format requirements of the target generation model.

[0187] It should be noted that the present application does not limit the implementation method of the "update process". For example, in some application scenarios, the "update process" can be implemented by rewriting. Based on this, it can be seen that in a possible implementation method, the above step 31 can be specifically: after obtaining the target generation model corresponding to the above target text, the target text is rewritten according to the text format constraint information corresponding to the target generation model to obtain the above updated text, so that the updated text can represent the text obtained by rewriting the target text according to the text format constraint information corresponding to the target generation model.

[0188] Based on the relevant content of step 31 above, it can be known that for some application scenarios, after obtaining the target generation model corresponding to the target text, the target text can be updated according to the text format constraint information corresponding to the target generation model to obtain an updated text, so that the updated text can express the semantic information expressed by the target text on the premise of satisfying the text format described by the text format constraint information, thereby enabling the updated text to express the semantic information expressed by the target text in accordance with the text format requirements of the target generation model.

[0189] Step 32: Use the above target generation model to perform image generation processing on the updated text to obtain image data corresponding to the target text.

[0190] In the present application, after obtaining the updated text corresponding to the above target text, the updated text can be directly input into the target generation model corresponding to the target text, so that the target generation model can perform image generation processing on the updated text, and obtain and output the image data corresponding to the target text. Among them, because the updated text can express the semantic information expressed by the target text according to the text format requirements of the target generation model, so that the updated text is more compatible with the target generation model, so that the image data generated for the updated text using the target generation model can better conform to the semantic information expressed by the target text, which is conducive to improving the image generation effect.

[0191] Based on the relevant contents of S1 to S3 above, it can be known that after obtaining the target text, the pre-constructed model tree is first used to determine the target generation model corresponding to the target text from at least two candidate generation models, so that the target generation model can represent the model that exists in the at least two candidate generation models and is more suitable for image generation processing for the target text; then the target generation model is used to generate image data corresponding to the target text, so that the image data can be as consistent as possible with the semantic information expressed by the target text, thereby achieving the purpose of generating image data from text.

[0192] In addition, because the above model tree is used to describe the category affiliation status of at least two candidate generation models above, the model tree can better represent the characteristics of different candidate generation models, so that the target generation model determined by the subsequent use of the model tree for a certain text (for example, the target text above) is more suitable for image generation processing for the text, which can effectively avoid defects caused by the incompatibility between the text and the image generation model (for example, defects such as poor image generation effect), thereby helping to improve the image generation effect.

[0193] In addition, the present application does not limit the execution subject of the image generation method provided in the embodiment of the present application. For example, the image generation method provided in the embodiment of the present application can be applied to a terminal device or a server. For another example, the image generation method provided in the embodiment of the present application can also be implemented with the help of the data interaction process between the terminal device and the server. Among them, the terminal device can be a smart phone, a computer, a personal digital assistant (PDA), a tablet computer, etc. The server can be a stand-alone server, a cluster server, or a cloud server.

[0194] Based on the image generation method provided in the embodiment of the present application, the embodiment of the present application also provides an image generation device. Figure 4 Explain and illustrate. Figure 4 This is a schematic diagram of the structure of an image generating device provided in an embodiment of the present application. It should be noted that for the technical details of the image generating device provided in an embodiment of the present application, please refer to the relevant contents of the image generating method above.

[0195] like Figure 4 As shown, the image generating device 400 provided in the embodiment of the present application includes:

[0196] An acquisition unit 401 is used to acquire a target text;

[0197] A determination unit 402 is used to determine a target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is used to describe the category attribution status of the at least two candidate generation models;

[0198] The generating unit 403 is used to generate image data corresponding to the target text by using the target generation model.

[0199] In one possible implementation, different leaf nodes in the model tree are used to represent different candidate generation models; for any leaf node in the model tree, the direct node of the leaf node is used to represent the category to which the candidate generation model represented by the leaf node belongs.

[0200] In a possible implementation manner, the determining unit 402 includes:

[0201] A first search subunit, used for searching the model tree for non-leaf nodes matching the target text to obtain a matching result;

[0202] The first determination subunit is used to determine a target generation model corresponding to the target text according to the matching result.

[0203] In a possible implementation manner, the first search subunit includes:

[0204] A core content extraction subunit is used to perform core content extraction processing on the target text to obtain the core content;

[0205] The second search subunit is used to search the model tree for non-leaf nodes that match the core content to obtain matching results.

[0206] In one possible implementation, the core content extraction subunit is specifically used to: determine a target style that matches the target text from extracted text styles corresponding to at least two core content extraction algorithms; and perform core content extraction processing on the target text using the core content extraction algorithm corresponding to the target style to obtain the core content.

[0207] In one possible implementation, the second search sub-unit is specifically used to: perform key content extraction processing on the core content to obtain at least one key content; search for non-leaf nodes matching the at least one key content from the model tree to obtain a matching result.

[0208] In a possible implementation manner, the model tree includes J non-leaf node aggregation layers arranged in sequence, where J is a positive integer; each of the non-leaf node aggregation layers includes at least two non-leaf nodes;

[0209] The first search sub-unit is specifically used to: search for nodes matching the target text from the first non-leaf node aggregation layer, and obtain the target node corresponding to the first non-leaf node aggregation layer; search for nodes matching the target text from the search range corresponding to the j-th non-leaf node aggregation layer, and obtain the target node corresponding to the j-th non-leaf node aggregation layer; the search range corresponding to the j-th non-leaf node aggregation layer is determined based on the target node corresponding to the j-1-th non-leaf node aggregation layer; j is a positive integer, 2≤j≤J; the matching result is determined based on the target nodes corresponding to the J non-leaf node aggregation layers.

[0210] In a possible implementation manner, the parent node of each node in the search range corresponding to the j-th non-leaf node aggregation layer is the corresponding target node of the j-1-th non-leaf node aggregation layer.

[0211] In one possible implementation, the first determination subunit is specifically used to: determine at least one candidate leaf node corresponding to the matching result from the model tree; and determine the target generation model corresponding to the target text based on data representing the degree of matching between the target text and the model description information corresponding to each of the candidate leaf nodes.

[0212] In a possible implementation manner, the model description information includes a model name and at least one model attribute.

[0213] In a possible implementation manner, the model tree is constructed based on model description information of the at least two candidate generation models, and the model description information includes a model name and at least one model attribute.

[0214] In one possible implementation, the model tree includes at least one non-leaf node aggregation layer and a leaf node aggregation layer; different nodes in the leaf node aggregation layer are used to represent the model names of different candidate generation models; the categories represented by the nodes in the different non-leaf node aggregation layers are determined based on the classification results of the at least two candidate generation models under different attribute dimensions.

[0215] In one possible implementation, the generation unit 403 is specifically used to: update the target text according to the text format constraint information corresponding to the target generation model to obtain updated text; and perform image generation processing on the updated text using the target generation model to obtain image data corresponding to the target text.

[0216] In a possible implementation manner, the text format constraint information corresponding to the target generation model includes a processed text sample corresponding to the target generation model.

[0217] Based on the relevant content of the above-mentioned image generating device 400, it can be known that for the image generating device 400 provided in the embodiment of the present application, after obtaining the target text, the pre-constructed model tree is first used to determine the target generation model corresponding to the target text from at least two candidate generation models, so that the target generation model can represent the model that exists in the at least two candidate generation models and is more suitable for image generation processing for the target text; and then the target generation model is used to generate image data corresponding to the target text, so that the image data can be as consistent as possible with the semantic information expressed by the target text, thereby achieving the purpose of generating image data from text.

[0218] Based on the image generation method provided in the embodiment of the present application, the embodiment of the present application also provides an image generation system. Figure 5 Explain and illustrate. Figure 5 This is a schematic diagram of the structure of an image generation system provided in an embodiment of the present application. It should be noted that for the technical details of the image generation system provided in an embodiment of the present application, please refer to the relevant content of the image generation method above.

[0219] like Figure 5 As shown, an image generation system 500 provided in an embodiment of the present application includes a controller 501 and at least two candidate generation models; wherein the controller 501 is used to execute any implementation of the image generation method provided in the present application. For ease of understanding, the following is an explanation with reference to examples.

[0220] As an example, in one possible implementation, the controller 501 is used to obtain a target text; the controller 501 is also used to determine a target generation model corresponding to the target text from at least two candidate generation models using a pre-built model tree; the model tree is used to describe the category attribution status of the at least two candidate generation models; the controller 501 is also used to call the target generation model to obtain image data corresponding to the target text; the target generation model is used to generate image data corresponding to the target text.

[0221] In a possible implementation manner, the controller 501 is a language model.

[0222] Based on the relevant content of the image generation system 500 above, it can be known that for the image generation system 500 provided in the embodiment of the present application, after obtaining the target text, the pre-constructed model tree is first used to determine the target generation model corresponding to the target text from at least two candidate generation models, so that the target generation model can represent the model that exists in the at least two candidate generation models and is more suitable for image generation processing for the target text; then, by calling the target generation model, the image data corresponding to the target text generated by the target generation model is obtained, so that the image data can be as consistent as possible with the semantic information expressed by the target text, thereby achieving the purpose of generating image data from text.

[0223] It can be seen that for the image generation system provided by the present application, the image generation system can generate high-quality images by searching for the generation model that best matches a certain text content, so that the image generation system can have the advantages of high versatility, high practicality and more convenience. In addition, the image generation system can be composed of a language model and a plurality of candidate generation models, so that the language model in the image generation system can act as a core controller to maintain the entire workflow of the system (for example, core content extraction, construction and search model thought tree, execution image generation and other processes), so that the image generation system can balance the advantages of general image generation models and image generation models in certain specific fields with the help of the language model, and automatically find the most matching model for any text content (for example, the text content input by the user) for image generation processing, so that the system can more flexibly achieve the purpose of generating high-quality images from text. In addition, the image generation system can more accurately extract the generation target described by any text content with the help of multiple core content extraction algorithms, which is conducive to improving the image generation effect. In addition, the image generation system can determine the model that best matches the text content by first searching the text content layer by layer in the model tree and then performing a refined search based on multiple attributes of the model. This is helpful in improving the accuracy of model matching and thus improving the image generation effect.

[0224] In addition, an embodiment of the present application also provides an electronic device, which includes a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes any implementation of the image generation method provided in the embodiment of the present application.

[0225] See also Figure 6 , which shows a schematic diagram of the structure of an electronic device 600 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0226] like Figure 6As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0227] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0228] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0229] The electronic device provided by the embodiment of the present disclosure and the method provided by the above embodiment belong to the same inventive concept, and the technical details not fully described in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0230] The embodiment of the present application also provides a computer-readable medium, in which instructions or computer programs are stored. When the instructions or computer programs are executed on a device, the device executes any implementation of the image generation method provided in the embodiment of the present application.

[0231] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0232] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0233] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0234] The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device can execute the method.

[0235] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0236] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0237] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit / module does not, in some cases, constitute a limitation on the unit itself.

[0238] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0239] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0240] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the system or device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description.

[0241] It should be understood that in the present application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0242] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0243] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0244] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An image generation method, characterized in that: The method comprises: Get the target text; Determine the target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is used to describe the category attribution status of the at least two candidate generation models; The target generation model is used to generate image data corresponding to the target text.

2. The method according to claim 1, characterized in that: Different leaf nodes in the model tree are used to represent different candidate generation models; For any leaf node in the model tree, the direct node of the leaf node is used to represent the category to which the candidate generation model represented by the leaf node belongs.

3. The method according to claim 2, characterized in that The process of determining the target generation model includes: Searching the model tree for non-leaf nodes that match the target text to obtain a matching result; According to the matching result, a target generation model corresponding to the target text is determined.

4. The method according to claim 3, characterized in that The process of determining the matching result includes: Performing core content extraction processing on the target text to obtain core content; The non-leaf nodes matching the core content are searched from the model tree to obtain matching results.

5. The method according to claim 4, characterized in that The process of determining the core content includes: Determining a target style matching the target text from among the extracted text styles corresponding to at least two core content extraction algorithms; The core content extraction algorithm corresponding to the target style is used to perform core content extraction processing on the target text to obtain the core content.

6. The method according to claim 4, characterized in that Before searching the model tree for non-leaf nodes matching the core content and obtaining a matching result, the method further includes: Performing key content extraction processing on the core content to obtain at least one key content; The step of searching the model tree for non-leaf nodes matching the core content to obtain matching results includes: The model tree is searched for non-leaf nodes matching the at least one key content to obtain a matching result.

7. The method according to claim 3, characterized in that The model tree includes J non-leaf node aggregation layers arranged in sequence, where J is a positive integer; each non-leaf node aggregation layer includes at least two non-leaf nodes; The process of determining the matching result includes: Searching for nodes matching the target text from the first non-leaf node aggregation layer to obtain a target node corresponding to the first non-leaf node aggregation layer; Searching for nodes matching the target text from the search range corresponding to the j-th non-leaf node aggregation layer to obtain the target node corresponding to the j-th non-leaf node aggregation layer; the search range corresponding to the j-th non-leaf node aggregation layer is determined based on the target node corresponding to the j-1-th non-leaf node aggregation layer; j is a positive integer, 2≤j≤J; The matching result is determined according to the target nodes corresponding to the J non-leaf node aggregation layers.

8. The method according to claim 7, characterized in that The parent node of each node in the search range corresponding to the j-th non-leaf node aggregation layer is the corresponding target node of the j-1-th non-leaf node aggregation layer.

9. The method according to claim 3, characterized in that: Determining the target generation model corresponding to the target text according to the matching result includes: Determine at least one candidate leaf node corresponding to the matching result from the model tree; The target generation model corresponding to the target text is determined based on the data representing the degree of matching between the target text and the model description information corresponding to each of the candidate leaf nodes.

10. The method according to claim 9, characterized in that The model description information includes a model name and at least one model attribute.

11. The method according to claim 1, characterized in that The model tree is constructed based on model description information of the at least two candidate generation models, wherein the model description information includes a model name and at least one model attribute; and / or, The model tree includes at least one non-leaf node aggregation layer and a leaf node aggregation layer; different nodes in the leaf node aggregation layer are used to represent the model names of different candidate generation models; the categories represented by the nodes in the different non-leaf node aggregation layers are determined based on the classification results of the at least two candidate generation models under different attribute dimensions.

12. The method according to claim 1, characterized in that The process of determining the image data corresponding to the target text includes: According to the text format constraint information corresponding to the target generation model, the target text is updated to obtain an updated text; The target generation model is used to perform image generation processing on the updated text to obtain image data corresponding to the target text.

13. The method according to claim 12, characterized in that The text format constraint information corresponding to the target generation model includes processed text samples corresponding to the target generation model.

14. An image generating device, characterized in that: include: An acquisition unit, used for acquiring a target text; A determination unit, configured to determine a target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is configured to describe the category attribution status of the at least two candidate generation models; A generating unit is used to generate image data corresponding to the target text by using the target generation model.

15. An image generation system, characterized in that: The image generation system includes a controller and at least two candidate generation models; The controller is used to obtain the target text; The controller is further used to determine the target generation model corresponding to the target text from at least two candidate generation models using a pre-constructed model tree; the model tree is used to describe the category attribution status of the at least two candidate generation models; The controller is further used to call the target generation model to obtain image data corresponding to the target text; the target generation model is used to generate image data corresponding to the target text.

16. The system according to claim 15, characterized in that The controller is a language model.

17. An electronic device, characterized in that: The device comprises: a processor and a memory; The memory is used to store instructions or computer programs; The processor is used to execute the instructions or computer programs in the memory so that the electronic device executes the method according to any one of claims 1 to 13.

18. A computer readable medium, characterized in that The computer-readable medium stores instructions or computer programs, and when the instructions or computer programs are executed on a device, the device executes the method according to any one of claims 1 to 13.

Images