Image generation method and device, computer storage medium and computer program product

Abstract

The embodiment of the invention provides an image generation method and device, a computer storage medium and a computer program product. The image generation method comprises the steps of obtaining target text data, and extracting key corpus data from the target text data; determining auxiliary description information of the key corpus data according to a pre-constructed corresponding relation library between the corpus data and the auxiliary description information; and inputting the key corpus data and the auxiliary description information of the key corpus data into a graph generation model, and generating image data corresponding to the target text data through the graph generation model. According to the embodiment of the invention, accurate image data with coherent content and high visual consistency can be efficiently generated under the condition that complex user input operation is not needed, and the image generation experience of the user is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular, to an image generation method, device, computer storage medium, and computer program product. Background Art

[0002] With the continuous development of science and technology, users have higher and higher requirements for reading experience. The traditional method of manually converting text into pictures can no longer meet the needs. Text-to-image, that is, the technology of automatically generating matching images based on text, has become a trend.

[0003] There are also a variety of text-generated image solutions provided in the related art. However, the above solutions usually generate images for text content of short sentences. For text content with relatively coherent content and longer length, such as novels, scripts, news, etc., in the text-generated image process, the user operation is complicated and the image set generation effect is not ideal, which affects the user experience. Summary of the invention

[0004] In view of this, an embodiment of the present application provides an image generation solution to at least partially solve the above-mentioned problem.

[0005] According to a first aspect of an embodiment of the present application, there is provided an image generating method, comprising:

[0006] Acquire target text data, and extract key corpus data from the target text data;

[0007] Determine the auxiliary description information of each key corpus data according to the pre-built correspondence database between the corpus data and the auxiliary description information;

[0008] The key corpus data and the auxiliary description information of the key corpus data are input into a raw image model, and image data corresponding to the target text data is generated by the raw image model.

[0009] According to a second aspect of an embodiment of the present application, there is provided an image generation method, which is applied to a client device, and the method includes:

[0010] Receive input target text data;

[0011] Sending the target text data to a server device so that the server device extracts key corpus data from the target text data, determines auxiliary description information of each key corpus data, and generates image data based on the key corpus data and the auxiliary description information;

[0012] Receive the image data returned by the server device.

[0013] According to the third aspect of an embodiment of the present application, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in the first aspect or the second aspect.

[0014] According to a fourth aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect or the second aspect above is implemented.

[0015] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising computer instructions, wherein the computer instructions instruct a computing device to perform operations corresponding to the method described in the first aspect or the second aspect above.

[0016] According to the image generation solution provided in the embodiment of the present application, after acquiring the target text data, key corpus data is extracted from the acquired target text data, and auxiliary description information of each key corpus data is determined according to a pre-constructed correspondence library between corpus data and auxiliary description information; then, each key corpus data and the auxiliary description information of the key corpus data are input into the raw image model, and image data corresponding to the target text data is generated by the raw image model.

[0017] On the one hand, the embodiments of the present application do not require the user to manually understand the target text data, rely on their own experience and writing skills to summarize the core semantic information contained in the target text data, and do not require the user to input the core semantic information to the image generation service through tedious and complicated input operations. Instead, after obtaining the target text data, an automated key corpus extraction operation is performed on the data to obtain the core semantic information contained in the data: key corpus data. Therefore, the embodiments of the present application can efficiently and accurately obtain key corpus information that can characterize the core semantic information of the target text data, so that the image generation operation can be performed based on the key corpus information in the future, so that the finally generated image data can more accurately match the core semantic information of the target text data.

[0018] On the other hand, more importantly, in the embodiment of the present application, a correspondence library between corpus data and auxiliary description information is pre-constructed. After the key corpus data is obtained, the auxiliary description information of each key corpus data is determined according to the above correspondence library; and then when the image is generated by the raw image model, in addition to using the extracted key corpus data as raw image reference information, the auxiliary description information of each key corpus data is additionally referenced, that is: the key corpus data and the auxiliary description information of the key corpus data are simultaneously used as raw image reference information. The above process, by pre-constructing a correspondence library, presets unified auxiliary description information for the same key corpus data as a supplementary description of the key corpus data. In this way, in the process of generating images based on key corpus data, more accurate, detailed and unified reference information can be provided to the raw image model to control the consistency of the elements in the image data, thereby ensuring the consistency of the content of the finally generated image data and the consistency of the visual experience.

[0019] In summary, the embodiments of the present application can efficiently generate accurate image data with coherent content and high visual consistency without complicated user input operations, thus effectively improving the user's image generation experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0021] Figure 1 is a flowchart of a method for generating an image according to Embodiment 1 of the present application;

[0022] Figure 2 A schematic diagram of an algorithm flow of an image generation method applicable to an embodiment of the present application;

[0023] Figure 3 A schematic diagram of the topological structure of an exemplary image generation system applicable to the image generation method of an embodiment of the present application;

[0024] Figure 4 A schematic diagram of the overall architecture of an exemplary image generation system applicable to the image generation method of an embodiment of the present application;

[0025] Figure 5 is a flowchart of a method for generating an image according to Embodiment 2 of the present application;

[0026] Figure 6 This is a schematic diagram of a scenario corresponding to the second embodiment of the present application;

[0027] Figure 7 is a structural block diagram of an image generating device according to Embodiment 3 of the present application;

[0028] Figure 8 is a structural block diagram of an image generating device according to Embodiment 4 of the present application;

[0029] Fig. 9 This is a schematic diagram of the structure of an electronic device according to Embodiment 5 of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0031] Embodiment 1

[0032] Reference Figure 1 , Figure 1 The following is a flowchart of a method for generating an image according to the first embodiment of the present application. The method for generating an image provided by the embodiment of the present application can be executed by any suitable device that is equipped with a raw image model. Specifically, the method for generating an image provided by the embodiment includes the following steps:

[0033] Step 102: Acquire target text data, and extract key corpus data from the target text data.

[0034] Specifically, the target text data in the embodiment of the present application can be any text data containing certain semantics. It can be text data containing more characters, that is, so-called long text data, such as: a complete novel, a chapter, a section, a script, news, etc.; it can also be text data containing fewer characters, such as: a monologue, a short dialogue, etc.

[0035] After obtaining the target text data, the key corpus data in the target text data can be extracted through step 102 of the embodiments of the present application, so as to obtain the key corpus data contained in the target text data. Among them, the key corpus data can be data representing the core semantic information of the target text data. Exemplarily, the key corpus data can include at least one of the following: character role data, scenario data, action data, gesture data, plot data. Optionally, in the embodiments of the present application, a neural network model can be used to perform the task of extracting key corpus data. Specifically: the process of extracting key corpus data from the target text data in this step can include:

[0036] Perform text preprocessing on the target text data to obtain processed text data; input the processed text data into a pre-trained key corpus extraction model, and output the key corpus data contained in the target text data through the key corpus extraction model.

[0037] Among them, the above text preprocessing process can include: operations such as removing stop words (such as words without actual meaning like "of", "and", etc.), word segmentation (decomposing the text into individual words or phrases), and part-of-speech tagging (such as nouns, verbs, adjectives, etc.).

[0038] In the above embodiments of the present application, a pre-trained key corpus extraction model is introduced to extract key corpus from the target text data. On the one hand, since the key corpus extraction model is pre-trained based on a large amount of sample data, therefore, by using the key corpus extraction model to extract key corpus, the accuracy of the extraction result can be guaranteed; on the other hand, in the actual extraction process, after the target text data is input into the key corpus extraction model, the key corpus extraction process can be automatically completed by the key corpus extraction model without manual participation. Therefore, the extraction efficiency is also relatively high. In summary, introducing a pre-trained key corpus extraction model to extract key corpus from the target text data can improve the accuracy of the extraction result while improving the extraction efficiency.

[0039] Step 104, determine the auxiliary description information of each key corpus data according to the pre-constructed correspondence library between the corpus data and the auxiliary description information.

[0040] Specifically, in the embodiments of the present application, for different corpus data, its corresponding auxiliary description information can be preset in advance, and then a correspondence library between the corpus data and the auxiliary description information is constructed. Among them, the auxiliary description information corresponding to the corpus data can be information used to supplement and refine the image corresponding to the corpus data. Through the auxiliary description information, more detailed information about the corpus data can be obtained. Exemplarily, for the corpus data "student", its corresponding auxiliary description information can be preset as: short hair, dark blue school uniform set, etc.

[0041] In the embodiments of the present application, there is no limitation on the method for generating the auxiliary description information corresponding to each corpus data. For example, it can be set manually based on experience, or generated with the help of a machine learning model, or first generated with the help of a machine learning model and then manually modified and edited the generated information, and so on.

[0042] Further, as described in step 102, the corpus data may include different types, such as: character data, scene data, action data, posture data, plot data, etc. Correspondingly, the auxiliary description information corresponding to the corpus data may also include multiple different types, wherein one type of corpus data corresponds to one type of auxiliary description information, for example: for corpus data of the type of character data, corresponding character auxiliary description information may be set for different characters, thereby obtaining a corresponding relationship library between characters and character auxiliary description information; for corpus data of the type of scene data, corresponding scene auxiliary description information may be set for different scenes, thereby obtaining a corresponding relationship library between scenes and scene auxiliary description information, etc. After the key corpus data is obtained through step 102, the auxiliary description information corresponding to each key corpus data may be searched and determined from the corresponding relationship library according to the type of the key corpus data.

[0043] Step 106: input the key corpus data and the auxiliary description information of the key corpus data into the raw image model, and generate image data corresponding to the target text data through the raw image model.

[0044] Specifically, the raw image model can be a pre-trained machine learning model used to generate corresponding image data based on text data. In the actual reasoning stage, the key corpus data and the auxiliary description information of the key corpus data can be used as reference information and input into the raw image model. After that, the raw image model can generate the desired image data that matches the semantics represented by the target text data based on the above reference information.

[0045] The image generation method provided in the embodiment of the present application, after acquiring the target text data, extracts the key corpus data from the acquired target text data, and also determines the auxiliary description information of each key corpus data according to a pre-constructed correspondence library between the corpus data and the auxiliary description information; then each key corpus data and the auxiliary description information of the key corpus data are input into the raw image model, and the image data corresponding to the target text data is generated by the raw image model.

[0046] On the one hand, the embodiments of the present application do not require the user to manually understand the target text data, rely on their own experience and writing skills to summarize the core semantic information contained in the target text data, and do not require the user to input the core semantic information to the image generation service through tedious and complicated input operations. Instead, after obtaining the target text data, an automated key corpus extraction operation is performed on the data to obtain the core semantic information contained in the data: key corpus data. Therefore, the embodiments of the present application can efficiently and accurately obtain key corpus information that can characterize the core semantic information of the target text data, so that the image generation operation can be performed based on the key corpus information in the future, so that the finally generated image data can more accurately match the core semantic information of the target text data.

[0047] On the other hand, more importantly, in the embodiment of the present application, a correspondence library between corpus data and auxiliary description information is pre-constructed. After the key corpus data is obtained, the auxiliary description information of each key corpus data is determined according to the above correspondence library; and then when the image is generated by the raw image model, in addition to using the extracted key corpus data as raw image reference information, the auxiliary description information of each key corpus data is additionally referenced, that is: the key corpus data and the auxiliary description information of the key corpus data are simultaneously used as raw image reference information. The above process, by pre-constructing a correspondence library, presets unified auxiliary description information for the same key corpus data as a supplementary description of the key corpus data. In this way, in the process of generating images based on key corpus data, more accurate, detailed and unified reference information can be provided to the raw image model to control the consistency of the elements in the image data, thereby ensuring the consistency of the content of the finally generated image data and the consistency of the visual experience.

[0048] In summary, the embodiments of the present application can efficiently generate accurate image data with coherent content and high visual consistency without complicated user input operations, thus effectively improving the user's image generation experience.

[0049] Optionally, in some of the embodiments, the image generating method may further include:

[0050] Get corpus data;

[0051] Inputting the corpus data and the target text data into the description information generation model, extracting semantic features from the corpus data and the target text data through the description information generation model, and predicting auxiliary description information based on the extracted semantic features;

[0052] The corpus data and the auxiliary description information predicted by the description information generation model are used to construct a corresponding relationship library between the corpus data and the auxiliary description information.

[0053] Specifically, after acquiring the corpus data, the above-mentioned embodiment of the present application predicts the auxiliary description information corresponding to the corpus data by means of a machine learning model combined with the semantic features of the corpus data in the target text data, and then constructs a corresponding relationship library between the corpus data and the auxiliary description information. Therefore, it can be ensured that each auxiliary description information predicted can more accurately reflect the image of the corresponding corpus data in the target text data, that is, it can effectively improve the accuracy of the corresponding relationship library finally constructed, and then the finally generated image data can be closer to the target text data at the semantic level.

[0054] In the embodiment of the present application, when constructing the corresponding relationship library, there is no limitation on the method of obtaining the corpus data, and the setting can be customized according to the actual situation. For example, the corpus data can be multiple data set manually based on experience, or multiple data parsed from a large amount of text data by artificial intelligence means, and so on.

[0055] Furthermore, in order to ensure that each key corpus data extracted from the target text data can find corresponding auxiliary description information in the constructed correspondence library, that is: in order to ensure that the constructed correspondence library can cover each key corpus data contained in the current target text data, or the key corpus data extracted from the target text data through the above step 102 can be used as the corpus data in the correspondence library, and then the auxiliary description information corresponding to each corpus data is generated through the description information generation model.

[0056] Optionally, in some of the embodiments, after building a corresponding relationship library between the corpus data and the auxiliary description information, the method further includes:

[0057] receiving modified auxiliary description information, where the modified auxiliary description information is information generated by a preset operation on the auxiliary description information in the corresponding relationship library;

[0058] The modified auxiliary description information is used to update the corresponding relationship library between the corpus data and the auxiliary description information.

[0059] Specifically, with respect to the correspondence library constructed in an automated manner, the auxiliary description information constructed for the corpus data may be somewhat different from the image that the user subjectively wants to present. In order to avoid the situation in which the image data finally generated cannot meet the user's expectations, in the above embodiment of the present application, after the correspondence library is obtained in an automated manner, the auxiliary description information corresponding to the corpus data can be displayed to the user; when a preset operation of the user on the displayed auxiliary description information is detected (the operation can be customized according to actual conditions), the modified auxiliary description information after the auxiliary description information is modified is determined, and then the auxiliary description information in the correspondence library is updated to the modified auxiliary description information.

[0060] The above-mentioned embodiment of the present application, after automatically obtaining the correspondence library between the corpus data and the auxiliary description information, also allows the user to edit and modify the auxiliary description information therein, so that the auxiliary description information in the corresponding relationship library finally established is closer to the image that the user mainly wants to present, thereby making the finally generated image data closer to the user's expectations.

[0061] Optionally, in some of the embodiments, after acquiring the target text data, the image generation method may further include:

[0062] Input the target text data into the copy extraction model, and obtain the copy outline data corresponding to the target text data through the copy extraction model;

[0063] Extract key corpus data from the target text data, including:

[0064] The document outline data is input into the corpus extraction model, and the key corpus data is extracted from the document outline data through the corpus extraction model.

[0065] Specifically, in some scenarios (such as long novels), the target text data (such as long dialogues) may contain a lot of redundant information. If key corpus extraction is performed on the target text data after obtaining it, a lot of redundant key corpus data will be obtained. For the image generation process, the redundant key corpus data will reduce the efficiency of image generation, but will not improve the quality of the final generated image data.

[0066] Based on the above analysis, in the embodiment of the present application, after acquiring the target text data, the target text data is first subjected to text extraction, thereby obtaining the text outline data corresponding to the target text data. The text outline data may be the simplified text data obtained after de-redundancy processing of the target text data. Performing subsequent image generation operations based on the simplified text data can improve the image generation efficiency while ensuring the image generation quality.

[0067] Optionally, in some of the embodiments, after acquiring the target text data, the method further includes:

[0068] Segmenting the target text data into a plurality of sentence text data;

[0069] Extract key corpus data from the target text data, including:

[0070] Extracting key corpus data corresponding to each sentence text data from each sentence text data;

[0071] Inputting key corpus data and auxiliary description information of the key corpus data into the raw image model, and generating image data corresponding to the target text data through the raw image model, including:

[0072] The key corpus data corresponding to each sentence text data and the auxiliary description information of each key corpus data are input into the raw image model, and the image data corresponding to each sentence text data is generated by the raw image model.

[0073] Specifically, in a long text scenario, the target text data usually contains many sentences. The above embodiment of the present application performs sentence text segmentation on the target text data, thereby performing subsequent image generation operations on each sentence text data, and then generating image data corresponding to each sentence text data. The above method, through the segmentation of the target text data, and the separate image generation operations on each sentence text data after segmentation, can make the finally generated image data closer to the corresponding sentence text data in terms of text semantics, and at the same time, make each image data have stronger coherence in text content.

[0074] Optionally, in some of the embodiments, the process of segmenting the target text data into a plurality of sentence text data may specifically include:

[0075] Performing plot boundary detection on the target text data to determine the plot boundary transition points in the target text data; wherein the plot boundary transition points are sentences in the target text data that represent a plot transition;

[0076] Based on the episode boundary transition points, the target text data is segmented into a plurality of sentence text data.

[0077] Specifically, the target text data is subjected to plot boundary detection, that is, sentences that can represent the change of the story plot (that is, the above-mentioned plot boundary transition points, such as those representing time jumps, location changes, character perspective switches, emotional changes, etc.) are detected from the complete target text data. After the plot boundary transition points are obtained, the target text data can be segmented according to the plot boundary transition points. The specific segmentation method is: according to the order in which the plot boundary transition points appear in the target text data, the text data between two adjacent plot boundary transition points are divided into one sentence text data.

[0078] In the above embodiment, when the target text data is segmented into sentences, it is mainly performed with reference to the plot boundary transition points in the target text data. Since the plot boundary transition points are sentences that represent the transition of the story plot, the above method of performing sentence segmentation with reference to the plot boundary transition points can ensure that the text characters in the single sentence text data obtained by segmentation have a close semantic relationship with each other and have a relatively complete story plot after combination. Afterwards, the subsequent image generation operation is performed based on the above sentence text data with close semantic relationship and complete plot, so that the generated image data can be further made close to the corresponding sentence text data in terms of text semantics, and at the same time, each image data has a stronger coherence in text content.

[0079] In the embodiment of the present application, a machine learning model (such as a neural network, a support vector machine, a random forest, etc.) can be used to detect plot boundaries. Specifically, before using the model, the model can be trained with a data set that has been annotated with plot boundary transition points, so that the model has the ability to detect plot boundaries; in the application stage, the target text data can be input into the trained machine learning model, thereby obtaining the plot boundary transition points in the target text data.

[0080] Furthermore, in order to further improve the completeness of the story in the final sentence text data, after the above-mentioned machine learning model outputs the plot boundary transition point, the plot boundary transition point can be automatically adjusted based on user input or other algorithms to determine the accuracy of the final plot boundary transition point, thereby improving the plot completeness of the sentence text data finally segmented.

[0081] Optionally, in some of the embodiments, after acquiring the target text data, the image generation method further includes:

[0082] Input the target text data into a preset text classification model, perform text category prediction on the text data through the text classification model, and obtain a text category prediction result;

[0083] When the text category prediction result meets the preset text category filtering condition, the step of extracting key corpus data from the target text data is executed;

[0084] Inputting key corpus data and auxiliary description information of the key corpus data into the raw image model, and generating image data corresponding to the target text data through the raw image model, including:

[0085] Inputting key corpus data and auxiliary description information of the key corpus data into a raw image model, and generating candidate image data through the raw image model;

[0086] The candidate image data is input into a preset image classification model, and the image classification model is used to predict the image category of the candidate image data to obtain an image category prediction result;

[0087] When the image category prediction result meets the preset image category filtering condition, the candidate image data is determined as the image data corresponding to the target text data.

[0088] Specifically, after acquiring the target text data, the embodiment of the present application uses a text classification model to predict the text category of the data, and then when the prediction result meets the preset text category filtering conditions, the subsequent image generation operation is performed on it. Correspondingly, after the image data is generated, the image data that does not meet the preset conditions is filtered out by image category prediction and preset image category filtering conditions. In the above manner, data filtering is performed from both text data and image data at the same time, thereby ensuring that the image data finally generated is data that meets the conditions, which helps to improve the security of the image generation process.

[0089] In the embodiment of the present application, the specific structure and parameter setting of the text classification model and the image classification model are not limited, and any appropriate classification model can be selected according to the actual situation. In addition, in the embodiment of the present application, the preset text category filtering condition and the preset image category filtering condition can also be customized according to the actual situation, and the specific content of the preset text category filtering condition and the preset image category filtering condition is not limited here.

[0090] Optionally, in some embodiments, the text category prediction result may include: safe text data or sensitive text data. Sensitive text data may be text data containing preset sensitive words, for example, the preset sensitive words may be words containing negative connotations such as violence and depression, or the preset sensitive words may be words set by the user according to his / her own wishes that are not suitable for image generation, etc. Safe text data may be text data that does not contain the preset sensitive words.

[0091] Correspondingly, when the text category prediction result satisfies the preset text category filtering condition, the step of extracting key corpus data from the target text data may include:

[0092] When the text category prediction result is safe text data, the step of extracting key corpus data from the target text data is executed; when the text category prediction result is sensitive text data, the image generation process is stopped and sensitive data prompt information is output.

[0093] When the type of the target text data is safe text data, the subsequent image generation process can be executed. When the type of the target text data is sensitive text data, it means that the target text data contains sensitive words that are not suitable for subsequent image generation. At this time, the subsequent process is stopped and sensitive data prompt information is output to the user so that the user can delete sensitive words and perform other processing on the target text data.

[0094] The image category prediction results include: secure image data or sensitive image data;

[0095] When the image category prediction result meets the preset image category filtering condition, the candidate image data is determined as the image data corresponding to the target text data, including:

[0096] When the image category prediction result is safe image data, the candidate image data is determined as the image data corresponding to the target text data; when the image category prediction result is sensitive image data, the candidate image data is cleared and sensitive data prompt information is output.

[0097] Similar to the text data type, in the embodiment of the present application, the image category prediction result can also be divided into safe image data or sensitive image data. Among them, sensitive image data can be image data containing preset sensitive elements, for example: the above-mentioned preset sensitive elements can be elements that convey negative emotions such as violence and depression, or the preset sensitive elements can also be elements set by the user according to his own wishes that are not suitable for display in image data, etc. Safe image data can be image data that does not contain the above-mentioned preset sensitive elements.

[0098] When the type of the generated candidate image data is safe image data, it can be determined as the final image data; when the type of the generated candidate image data is sensitive image data, it means that the candidate image data contains sensitive elements that are not suitable for being included in the image data. At this time, the candidate image data can be cleared and sensitive data prompt information can be output to the user.

[0099] See also Figure 2 , Figure 2 The following is a schematic diagram of an algorithm flow chart of an image generation method applicable to an embodiment of the present application. Figure 2The image generation method provided in the above example 1 of this application is briefly described:

[0100] After obtaining the target text data input by the user, the target text data can be firstly subjected to security filtering, that is: the target text data is input into a preset text classification model, and the text classification model is used to predict the text category of the text data to obtain the text category prediction result; when the text category prediction result meets the preset text category filtering condition, the corpus extraction model is used to extract the key corpus of the target text data to obtain the key corpus data; according to the pre-constructed correspondence library between the corpus data and the auxiliary description information, the auxiliary description information of each key corpus data is determined; the key corpus data and the auxiliary description information of the key corpus data are input into the raw image model, and the image data is generated by the raw image model; thereafter, in order to ensure that the final image data meets the security requirements, the generated image data can be subjected to security filtering: the generated image data is input into a preset image classification model, and the image classification model is used to predict the image category of the image data to obtain the image category prediction result; when the image category prediction result meets the preset image category filtering condition, the generated image data is output.

[0101] See also Figure 3 , Figure 3 This is a schematic diagram of the topological structure of an exemplary image generation system applicable to the image generation method of the embodiment of the present application. For ease of understanding, first combine Figure 3 The application scenario of the image generation method provided in the embodiment of the present application is explained.

[0102] like Figure 3 As shown, the system 300 may include a server 302, a communication network 304 and / or one or more user devices (ie, client devices) 306. Figure 3 The example in FIG. 1 is a plurality of user devices.

[0103] The server 302 may be any suitable device that is deployed with a raw image model for image generation, including but not limited to a distributed storage system device, a server cluster, a computing cloud server cluster, etc. In some embodiments, the server 302 may perform any suitable function. For example, in some embodiments, the server 302 may be used to generate image data corresponding to the target text data through the raw image model, and send it to the corresponding user device 306 for the user to view and use.

[0104] In some embodiments, the communication network 304 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 304 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN) and / or any other suitable communication network. The user device 306 can be connected to the communication network 304 via one or more communication links (e.g., communication link 312), and the communication network 304 can be linked to the server 302 via one or more communication links (e.g., communication link 314). The communication link can be any communication link suitable for transmitting data between the user device 306 and the server 302, such as a network link, a dial-up link, a wireless link, a hard-wired link, any other suitable communication link, or any suitable combination of such links.

[0105] User equipment 306 may include any one or more user equipment suitable for image display. In some embodiments, user equipment 306 may include any suitable type of equipment. For example, in some embodiments, user equipment 306 may include a mobile device, a tablet computer, a laptop computer, a desktop computer, and / or any other suitable type of user equipment.

[0106] Based on the above system, the image generation method provided in the above embodiment 1 of the present application can be Figure 3 The server 302 in the system shown executes. Specifically:

[0107] Step 102 may include: receiving target text data sent by the client device, and extracting key corpus data from the target text data. Correspondingly, after generating image data corresponding to the target text data in step 106, it also includes: returning the image data to the client device so that the client device displays the image data.

[0108] See also Figure 4 , Figure 4 FIG. 1 is a schematic diagram of the overall architecture of an exemplary image generation system applicable to the image generation method of the embodiment of the present application. Figure 4 The overall architecture of the above image generation system is explained as follows:

[0109] From an architectural perspective, the image generation system provided in the embodiment of the present application may mainly include five parts: presentation layer, communication layer, service layer, algorithm layer, and storage layer.

[0110] The presentation layer may be located on the user device 306, and is used to receive and display the target text data input by the user, and receive and display the image data returned by the service layer. Exemplarily, the presentation layer may send a network request to the service layer through the Web client and the Restful interface in the user device 306, and with the help of the communication layer, so as to interact with the service layer for data.

[0111] The communication layer can be implemented by various network request methods, for example, HSF (High-speed Service Framework), Socket, HTTP / HTTPS, etc. can be used for network request.

[0112] The service layer can be located on the server device. The service layer in the embodiment of the present application provides service interfaces related to the image generation service, such as, for example, a character preview interface, a drawing interface, a feature extraction interface, a progress acquisition interface, and the like. Among them, the feature extraction interface is used to call the feature extraction service, and the physical sign extraction service is used to extract key corpus data from the target text data; the drawing interface is used to call the drawing service, and the drawing service is used to determine the auxiliary description information of each key corpus data, and generate image data based on the key corpus data and the auxiliary description information; the character preview interface is used to feedback the generated image data to the user for user preview; the progress acquisition interface is used to call the progress acquisition service, and the progress acquisition service is used to feedback the current drawing progress information to the user during the drawing process. In addition, the service layer in the embodiment of the present application also introduces monitoring protection (including such as Figure 4 It uses multiple components such as link tracking, cluster monitoring, fault-tolerant protection, gateway, governance configuration, service governance, etc. to ensure the stability, scalability, and compatibility of services related to image generation.

[0113] The algorithm layer is also located on the server device. The algorithm layer mainly implements core operations such as preprocessing of target text data, data processing (for example, such as word segmentation, part-of-speech tagging, etc.), machine translation (translating the key corpus information extracted from the target text data and the auxiliary description information corresponding to the key corpus data obtained by the query into a language that the raw image model can understand, and then, after the raw image model generates image data, the image data is translated into a language that the user can understand), real-time calculation (extracting key corpus information from the target text data, etc.), and drawing.

[0114] The storage layer may be located on the server device or in a storage device independent of the server device. Figure 4 As shown, the storage layer may include databases (such as MySQL databases), caches (such as node caches), file storage services (such as Object Storage Service, object storage services; highly available distributed KV (Key-Value) storage engines), etc., which are used to store data resources generated by the algorithm layer and improve resource acquisition performance.

[0115] Based on the above image generation system, the embodiment of the present application also provides an image generation method, which can be Figure 3 The user equipment 306 in the system shown in the figure executes the above operation. The following describes the operation through multiple embodiments.

[0116] Embodiment 2

[0117] Reference Figure 5 , Figure 5 is a flowchart of the steps of an image generation method according to the second embodiment of the present application. The image generation method provided in the embodiment of the present application can be Figure 3 The user device 306 in the system shown in the figure executes. Specifically, the image generation method provided in this embodiment includes the following steps:

[0118] Step 502: receiving input target text data.

[0119] Specifically, the user device 306 can receive the target text data input by the user. In the embodiment of the present application, the specific method used by the user device 306 to receive the target text data is not limited, for example: a text input box can be displayed to the user, and the user can input the target text data into the text data box; or the user's voice input information can be detected and the voice input information can be converted into text to obtain the target text data, etc.

[0120] The target text data in the embodiment of the present application can be any text data containing certain semantic information. It can be text data containing more characters, that is, so-called long text data, such as: a complete novel, a chapter, a section, a script, news, etc.; it can also be text data containing fewer characters, such as: a monologue, a short dialogue, etc.

[0121] Step 504, sending the target text data to the server device, so that the server device extracts key corpus data from the target text data, determines auxiliary description information of each key corpus data, and generates image data based on the key corpus data and the auxiliary description information.

[0122] Specifically, after the user device 306 sends the target text data to the server device, the server device can call the preset feature extraction interface to extract key corpus data from the target text data, and then call the preset drawing interface to determine the auxiliary description information of each key corpus data, and generate image data based on the key corpus data and the auxiliary description information.

[0123] Step 506: Receive the image data returned by the server device.

[0124] In the image generation method provided by the embodiment of the present application, after receiving the target text data input by the user, the client device sends the target text data to the server device, and the server device automatically performs subsequent image generation operations and returns the produced image data. For the user, the operation to be performed is to input the target text data into the client, without the need for the user to manually understand the target text data, rely on their own experience and writing skills to summarize the core semantic information contained in the target text data, and without the need for the user to input the core semantic information to the image generation service through tedious and complicated input operations. Therefore, the embodiment of the present application can reduce the complexity of user operations, that is, reduce the difficulty of getting started, and therefore, can improve the user's experience of using raw images.

[0125] In addition, after acquiring the target text data, the server-side device performs an automated key corpus extraction operation on the data to obtain the core semantic information contained in the data: key corpus data. Therefore, the embodiment of the present application can efficiently and accurately acquire key corpus information that can characterize the core semantic information of the target text data, so that subsequent image generation operations can be performed based on the key corpus information, so that the finally generated image data can more accurately match the core semantic information of the target text data.

[0126] Moreover, more importantly, in the embodiment of the present application, the server-side device also pre-constructs a correspondence library between the corpus data and the auxiliary description information. After acquiring the key corpus data, the auxiliary description information of each key corpus data is determined according to the above correspondence library; and then when generating an image with the help of the raw image model, in addition to using the extracted key corpus data as raw image reference information, the auxiliary description information of each key corpus data is additionally referenced, that is: the key corpus data and the auxiliary description information of the key corpus data are simultaneously used as raw image reference information. The above process, by pre-constructing a correspondence library, presets unified auxiliary description information for the same key corpus data as a supplementary description of the key corpus data. In this way, in the process of generating images based on key corpus data, more accurate, detailed and unified reference information can be provided to the raw image model to control the consistency of the elements in the image data, thereby ensuring the consistency of the content of the finally generated image data and the consistency of the visual experience.

[0127] In summary, the embodiments of the present application can efficiently generate accurate image data with coherent content and high visual consistency without complicated user input operations, thus effectively improving the user's image generation experience.

[0128] Optionally, in some of the embodiments, the above image generating method may further include:

[0129] Receive auxiliary description information corresponding to the corpus data sent by the server device, and display the auxiliary description information;

[0130] In response to a modification operation on the auxiliary description information, determining the auxiliary description information after the modification;

[0131] The modified auxiliary description information is returned to the server device, so that the server device updates the pre-constructed correspondence library between the corpus data and the auxiliary description information based on the modified auxiliary description information.

[0132] Specifically, with respect to the correspondence library constructed by the server device in an automated manner, the auxiliary description information constructed for the corpus data may differ to a certain extent from the image that the user subjectively wants to present. In order to avoid the finally generated image data being unable to meet the user's expectations due to the above situation, in the above embodiment of the present application, after obtaining the correspondence library in an automated manner, the server device may also return the auxiliary description information corresponding to the corpus data to the client device, and the client device displays the above auxiliary description information; when the client device detects the user's preset operation on the displayed auxiliary description information (the operation can be customized according to actual conditions), the modified auxiliary description information after the modification of the auxiliary description information is determined and reported to the server device, thereby enabling the server device to update the auxiliary description information in the correspondence library to the modified auxiliary description information.

[0133] The above-mentioned embodiment of the present application, after automatically obtaining the correspondence library between the corpus data and the auxiliary description information, also allows the user to edit and modify the auxiliary description information therein, so that the auxiliary description information in the corresponding relationship library finally established is closer to the image that the user mainly wants to present, thereby making the finally generated image data closer to the user's expectations.

[0134] Optionally, in some of the embodiments, after sending the target text data to the server device, the method further includes:

[0135] Receive key corpus data sent by the server device;

[0136] In response to a modification operation on the key corpus data, determining the modified key corpus data;

[0137] The modified key corpus data is returned to the server device, so that the server device determines the auxiliary description information based on the modified key corpus data, and generates image data based on the key corpus data and the auxiliary description information.

[0138] Specifically, similar to the auxiliary description information in the corresponding relationship library, the key corpus data extracted by the server device from the target text data in an automated manner may also be somewhat different from the content that the user subjectively wants to highlight in the image data. In order to avoid the final generated image data failing to meet user expectations due to the above situation, in the above embodiment of the present application, after obtaining the key corpus data in an automated manner, the server device may also return the key corpus data to the client device, and the client device displays the above key corpus data; when the client device detects the user's preset operation on the displayed key corpus data (the operation can be customized according to actual conditions), the modified key corpus data after the modification of the key corpus data is determined and reported to the server device, so that the server device updates the key corpus data to the modified key corpus data.

[0139] The above-mentioned embodiment of the present application allows the user to edit and modify the key corpus data after the key corpus data is automatically extracted, so that the key corpus data is closer to the content that the user mainly wants to present, and thus the finally generated image data can be closer to the user's expectations.

[0140] Optionally, in some of the embodiments, the above image generating method may further include:

[0141] The target text data is displayed on the display page, and during the display of the target text data, the image data corresponding to the target text data is played.

[0142] Specifically, after the client device receives the image data generated for the target text data returned by the server device, the corresponding image data can be displayed synchronously during the display of the target text data, so as to enhance the reading experience and immersion of the reader of the target text data by converting the content in the target text data into image data.

[0143] See also Figure 6 , Figure 6 This is a schematic diagram of the scene corresponding to the second embodiment of the present application. Figure 6 The above-mentioned second embodiment is explained as follows:

[0144] When a user wants to generate corresponding image data for a certain novel chapter, the user can input the text data of the novel chapter through the text data input interface provided in the client device; thereafter, by detecting the user clicking on the "text-to-image" icon in the page, the client device can send an HTTP request to the server device, and the request reaches the target server in the server cluster after being transferred through the request routing; the target service calls the corresponding raw image service through the preset raw image service interface, thereby providing the client device with the raw image function.

[0145] Specifically, Figure 6 As shown, the image generation service interface may include: image preview interface (character preview interface), image generation interface (drawing interface), progress display interface (progress acquisition interface), feature extraction interface, etc. The functions of the above interfaces have been Figure 4 The manual explains that I will not repeat it here.

[0146] In addition, in order to facilitate the storage of resource data and improve resource acquisition performance, the server cluster is also configured with a corresponding cache cluster, which is used to store the data resources generated by the server cluster during the image generation process.

[0147] Embodiment 3

[0148] Figure 7 : is a structural block diagram of an image generation device according to Embodiment 3 of the present application. The image generation device provided in the embodiment of the present application includes:

[0149] A text acquisition module 702 is used to acquire target text data and extract key corpus data from the target text data;

[0150] An information determination module 704 is used to determine the auxiliary description information of each key corpus data according to a pre-built correspondence database between corpus data and auxiliary description information;

[0151] The image generation module 706 is used to input the key corpus data and the auxiliary description information of the key corpus data into the image generation model, and generate image data corresponding to the target text data through the image generation model.

[0152] Optionally, in some of the embodiments, the image generating device further comprises:

[0153] Relational library building blocks for:

[0154] Get corpus data;

[0155] Inputting the corpus data and the target text data into the description information generation model, extracting semantic features from the corpus data and the target text data through the description information generation model, and predicting auxiliary description information based on the extracted semantic features;

[0156] The corpus data and the auxiliary description information predicted by the description information generation model are used to construct a corresponding relationship library between the corpus data and the auxiliary description information.

[0157] Optionally, in some of the embodiments, after constructing the correspondence library between the corpus data and the auxiliary description information, the relationship library construction module is also used to: receive modified auxiliary description information, the modified auxiliary description information being information generated by a preset operation on the auxiliary description information in the correspondence library; and update the correspondence library between the corpus data and the auxiliary description information using the modified auxiliary description information.

[0158] Optionally, in some embodiments, after acquiring the target text data, the text acquisition module 702 is further used to: input the target text data into a text extraction model, and obtain text outline data corresponding to the target text data through the text extraction model;

[0159] Correspondingly, when executing the step of extracting key corpus data from target text data, the text acquisition module 702 is specifically used to: input the text outline data into the corpus extraction model, and extract key corpus data from the text outline data through the corpus extraction model.

[0160] Optionally, in some of the embodiments, the key corpus data includes at least one of the following: character data, scene data, action data, posture data, plot data; the text acquisition module 702, when executing the step of extracting the key corpus data from the target text data, is specifically used to: perform text preprocessing on the target text data to obtain processed text data; input the processed text data into a pre-trained key corpus extraction model, and output the key corpus data contained in the target text data through the key corpus extraction model.

[0161] Optionally, in some of the embodiments, after acquiring the target text data, the text acquisition module 702 is further used to: segment the target text data into a plurality of sentence text data;

[0162] Correspondingly, the text acquisition module 702, when executing the step of extracting key corpus data from the target text data, is specifically used to: extract key corpus data corresponding to each sentence text data from each sentence text data;

[0163] The image generation module 706 is specifically used to input the key corpus data corresponding to each sentence text data and the auxiliary description information of each key corpus data into the image generation model, and generate the image data corresponding to each sentence text data through the image generation model.

[0164] Optionally, in some of the embodiments, when executing the step of segmenting the target text data into a plurality of sentence text data, the text acquisition module 702 is specifically used to: perform plot boundary detection on the target text data to determine the plot boundary transition points in the target text data; the plot boundary transition points are sentences in the target text data that represent the transition of the story plot; based on the plot boundary transition points, segment the target text data into a plurality of sentence text data.

[0165] Optionally, in some embodiments, after acquiring the target text data, the text acquisition module 702 is further used to: input the target text data into a preset text classification model, perform text category prediction on the text data through the text classification model, and obtain a text category prediction result; when the text category prediction result meets the preset text category filtering condition, perform the step of extracting key corpus data from the target text data;

[0166] Correspondingly, the image generation module 706 is specifically used to: input the key corpus data and the auxiliary description information of the key corpus data into the image generation model, and generate candidate image data through the image generation model; input the candidate image data into a preset image classification model, and perform image category prediction on the candidate image data through the image classification model to obtain an image category prediction result; when the image category prediction result meets the preset image category filtering condition, determine the candidate image data as the image data corresponding to the target text data.

[0167] Optionally, in some of the embodiments, the text category prediction result includes: safe text data or sensitive text data; the text acquisition module 702, when executing the step of extracting key corpus data from the target text data when the text category prediction result meets the preset text category filtering condition, is specifically used to: when the text category prediction result is safe text data, execute the step of extracting key corpus data from the target text data; when the text category prediction result is sensitive text data, stop the image generation process and output sensitive data prompt information;

[0168] Correspondingly, the image category prediction result includes: safe image data or sensitive image data; the image generation module 706, when executing the step of determining the candidate image data as the image data corresponding to the target text data when the image category prediction result meets the preset image category filtering condition, is specifically used for: when the image category prediction result is safe image data, determining the candidate image data as the image data corresponding to the target text data; when the image category prediction result is sensitive image data, clearing the candidate image data and outputting sensitive data prompt information.

[0169] The image generation device of this embodiment is used to implement the corresponding image generation method in the above-mentioned embodiment 1, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here. In addition, the functional implementation of each module in the image generation device of this embodiment can refer to the description of the corresponding part in the above-mentioned method embodiment 1, which will not be described in detail here.

[0170] Embodiment 4

[0171] Figure 8 : is a structural block diagram of an image generating device according to Embodiment 4 of the present application. The image generating device provided in the embodiment of the present application is applied to a client device, and the image generating device includes:

[0172] The text receiving module 802 is used to receive input target text data;

[0173] The sending module 804 is used to send the target text data to the server device, so that the server device extracts key corpus data from the target text data, determines auxiliary description information of each key corpus data, and generates image data based on the key corpus data and the auxiliary description information;

[0174] The image receiving module 806 is used to receive the image data returned by the server device.

[0175] Optionally, in some of the embodiments, the image generating device further comprises:

[0176] The auxiliary description information modification module is used to: receive the auxiliary description information corresponding to the corpus data sent by the server device and display the auxiliary description information; in response to the modification operation of the auxiliary description information, determine the auxiliary description information after the modification; return the modified auxiliary description information to the server device, so that the server device updates the pre-constructed correspondence library between the corpus data and the auxiliary description information based on the modified auxiliary description information.

[0177] Optionally, in some of the embodiments, the image generating device further comprises:

[0178] The key corpus data modification module is used to: after sending the target text data to the server device, receive the key corpus data sent by the server device; in response to the modification operation on the key corpus data, determine the modified key corpus data; return the modified key corpus data to the server device, so that the server device determines the auxiliary description information based on the modified key corpus data, and generates image data based on the key corpus data and the auxiliary description information.

[0179] Optionally, in some of the embodiments, the image generating device further comprises:

[0180] The display module is used to display the target text data in the display page, and play the image data corresponding to the target text data during the display of the target text data.

[0181] The image generation device of this embodiment is used to implement the corresponding image generation method in the aforementioned embodiment 2, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here. In addition, the functional implementation of each module in the image generation device of this embodiment can refer to the description of the corresponding part in the aforementioned method embodiment 2, which will not be described in detail here.

[0182] Embodiment 5

[0183] Reference Fig. 9 , shows a schematic diagram of the structure of an electronic device according to Embodiment 5 of the present application. The specific embodiments of the present application do not limit the specific implementation of the electronic device.

[0184] like Fig. 9 As shown, the electronic device may include: a processor (processor) 902 , a communication interface (Communications Interface) 904 , a memory (memory) 906 , and a communication bus 908 .

[0185] in:

[0186] The processor 902 , the communication interface 904 , and the memory 906 communicate with each other via a communication bus 908 .

[0187] The communication interface 904 is used to communicate with other electronic devices or servers.

[0188] The processor 902 is used to execute the program 910, and specifically can execute the relevant steps in the above-mentioned image generation method embodiment.

[0189] Specifically, the program 910 may include program codes, which include computer operation instructions.

[0190] The processor 902 may be a CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0191] The memory 906 is used to store the program 910. The memory 906 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0192] The program 910 may include multiple computer instructions. Specifically, the program 910 may enable the processor 902 to execute operations corresponding to the image generation method described in the above embodiments through the multiple computer instructions.

[0193] The specific implementation of each step in program 910 can refer to the corresponding description of the corresponding steps and units in the above method embodiment, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0194] The present application also provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in any of the above-mentioned multiple method embodiments is implemented. The computer storage medium includes but is not limited to: a compact disc read-only memory (CD-ROM), a random access memory (RAM), a floppy disk, a hard disk or a magneto-optical disk, etc.

[0195] An embodiment of the present application also provides a computer program product, including computer instructions, which instruct a computing device to execute operations corresponding to any one of the above-mentioned multiple method embodiments.

[0196] In addition, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used to train the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0197] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0198] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor or programmable or dedicated hardware (such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA)). It can be understood that a computer, a processor, a microprocessor controller or programmable hardware includes a storage component (e.g., a random access memory (RAM), a read-only memory (ROM), a flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown here, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown here.

[0199] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0200] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. A method for generating an image, comprising: Acquire target text data, and extract key corpus data from the target text data; Determine the auxiliary description information of each key corpus data according to the pre-built correspondence database between the corpus data and the auxiliary description information; The key corpus data and the auxiliary description information of the key corpus data are input into a raw image model, and image data corresponding to the target text data is generated by the raw image model.

2. The method according to claim 1, wherein: Also includes: Get corpus data; Inputting the corpus data and the target text data into a description information generation model, extracting semantic features from the corpus data and the target text data through the description information generation model, and predicting auxiliary description information based on the extracted semantic features; The corpus data and the auxiliary description information predicted by the description information generation model are used to construct a corresponding relationship library between the corpus data and the auxiliary description information.

3. The method according to claim 2, wherein: After constructing the corresponding relationship library between the corpus data and the auxiliary description information, the method further includes: receiving modified auxiliary description information, wherein the modified auxiliary description information is information generated by a preset operation on the auxiliary description information in the correspondence library; The modified auxiliary description information is used to update the corresponding relationship library between the corpus data and the auxiliary description information.

4. The method according to any one of claims 1 to 3, wherein: After acquiring the target text data, the method further includes: Inputting the target text data into a text extraction model, and obtaining text outline data corresponding to the target text data through the text extraction model; The step of extracting key corpus data from the target text data includes: The document outline data is input into a corpus extraction model, and key corpus data is extracted from the document outline data through the corpus extraction model.

5. The method according to any one of claims 1 to 3, wherein: The key corpus data includes at least one of the following: character data, scene data, action data, posture data, and plot data; Extracting key corpus data from the target text data includes: Performing text preprocessing on the target text data to obtain processed text data; The processed text data is input into a pre-trained key corpus extraction model, and the key corpus data contained in the target text data is output through the key corpus extraction model.

6. The method according to any one of claims 1 to 3, wherein: After acquiring the target text data, the method further includes: Segmenting the target text data into a plurality of sentence text data; The step of extracting key corpus data from the target text data includes: Extracting key corpus data corresponding to each sentence text data from each sentence text data; The step of inputting the key corpus data and the auxiliary description information of the key corpus data into a raw image model, and generating image data corresponding to the target text data through the raw image model, comprises: The key corpus data corresponding to each sentence text data and the auxiliary description information of each key corpus data are input into the raw image model, and the image data corresponding to each sentence text data is generated by the raw image model.

7. An image generation method, applied to a client device, the method comprising: Receive input target text data; Sending the target text data to a server device so that the server device extracts key corpus data from the target text data, determines auxiliary description information of each key corpus data, and generates image data based on the key corpus data and the auxiliary description information; Receive the image data returned by the server device.

8. An electronic device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1-7.

9. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer program product, comprising computer instructions, wherein the computer instructions instruct a computing device to perform operations corresponding to the method according to any one of claims 1 to 7.

Images