Text annotation model training method, text-generated graph method and device

Abstract

The present application discloses a training method for a text annotation model, a method and apparatus for a text annotation graph, and relates to the field of artificial intelligence technology, in particular to the fields of natural language processing and deep learning. Implementation scheme: obtaining a first sample text and multi-level reference labels corresponding to each character in the first sample text; annotating the first sample text using an initial text annotation model to obtain a predicted label sequence; training the initial text annotation model based on the difference between the predicted multi-level labels corresponding to each character and the multi-level reference labels to obtain a text annotation model. Thus, a text annotation model capable of multi-level labeling can be trained by annotating the first sample text with multi-level labels, annotating the input text of the text annotation graph with the text annotation model, and generating an image based on the input text annotated with multi-level labels, thereby improving the matching degree between the generated image and the description of the input text.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, in particular to the fields of natural language processing and deep learning, and specifically to a training method for a text annotation model, a text-generating graph method, and a device. Background Art

[0002] A text-based graph generates an image based on a text description. Related technologies use word segmentation models, part-of-speech tagging models, and proper name tagging models to annotate text. However, when text annotated using these models is applied to a text-based graph, the resulting image has a low degree of match with the text description. Summary of the Invention

[0003] This application provides a text annotation model training method, a text-to-graph method, and a device. The specific solution is as follows:

[0004] According to one aspect of the present application, a method for training a text annotation model is provided, comprising:

[0005] Obtaining a first sample text and multi-level reference labels corresponding to each character in the first sample text;

[0006] Annotate the first sample text using the initial text annotation model to obtain a predicted label sequence, where the predicted label sequence includes predicted multi-level labels corresponding to each character;

[0007] Training the initial text annotation model according to the difference between the predicted multi-level labels corresponding to each character and the multi-level reference labels to obtain a text annotation model;

[0008] The text annotation model is used to annotate the input text of the text-to-graph model, and the text-to-graph model is used to process the annotated input text to generate an image corresponding to the input text.

[0009] According to another aspect of the present application, a method for generating a cultural image is provided, comprising:

[0010] Get the target text;

[0011] Annotating the target text using a text annotation model to obtain a sequence annotation result corresponding to the target text, wherein the text annotation model is trained using the method described in the above-mentioned embodiment;

[0012] The target text and sequence annotation results are input into the text-to-graph model for processing to generate an image corresponding to the target text.

[0013] According to another aspect of the present application, a training device for a text annotation model is provided, comprising:

[0014] A first acquisition module is used to acquire a first sample text and a multi-level reference label corresponding to each character in the first sample text;

[0015] A second acquisition module is used to annotate the first sample text using the initial text annotation model to obtain a predicted label sequence, where the predicted label sequence includes predicted multi-level labels corresponding to each character;

[0016] A training module is used to train the initial text annotation model according to the difference between the predicted multi-level labels corresponding to each character and the multi-level reference labels to obtain a text annotation model;

[0017] The text annotation model is used to annotate the input text of the text-to-graph model, and the text-to-graph model is used to process the annotated input text to generate an image corresponding to the input text.

[0018] According to another aspect of the present application, a cultural image device is provided, comprising:

[0019] A first acquisition module is used to acquire the target text;

[0020] A second acquisition module is used to annotate the target text using a text annotation model to obtain a sequence annotation result corresponding to the target text, wherein the text annotation model is trained using the above-mentioned training method;

[0021] The generation module is used to input the target text and sequence annotation results into the text-graph model for processing to generate an image corresponding to the target text.

[0022] According to another aspect of the present application, an electronic device is provided, including:

[0023] at least one processor; and

[0024] a memory communicatively connected to the at least one processor; wherein,

[0025] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to execute the method described in the embodiment of one aspect or the method described in the embodiment of another aspect.

[0026] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method according to the above-mentioned one embodiment or the method according to the above-mentioned other embodiment.

[0027] According to another aspect of the present application, a computer program product is provided, comprising a computer program, which implements the steps of the method described in the embodiment of the first aspect when executed by a processor, or implements the steps of the method described in the embodiment of the other aspect when executed.

[0028] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present application.

[0030] Figure 1 A flowchart of a method for training a text annotation model provided in one embodiment of the present application;

[0031] Figure 2 A flowchart of a method for training a text annotation model provided in another embodiment of the present application;

[0032] Figure 3 A flowchart of a method for training a text annotation model provided in another embodiment of the present application;

[0033] Figure 4 A flowchart of a method for training a text annotation model provided in another embodiment of the present application;

[0034] Figure 5 A schematic diagram of a process flow of a text graph method provided in one embodiment of the present application;

[0035] Figure 6 A schematic diagram of the structure of a training device for a text annotation model provided in one embodiment of the present application;

[0036] Figure 7 A schematic structural diagram of a wenshengtu device provided in one embodiment of the present application;

[0037] Figure 8 It is a block diagram of an electronic device used to implement the training method of the text annotation model of the embodiment of the present application. DETAILED DESCRIPTION

[0038] The following description of exemplary embodiments of the present application is made in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0039] The following describes the text annotation model training method, text-to-graph method, and apparatus according to embodiments of the present application with reference to the accompanying drawings.

[0040] Figure 1 A flowchart of a method for training a text annotation model provided in one embodiment of the present application.

[0041] The training method of the text annotation model of the embodiment of the present application can be executed by the training device of the text annotation model of the embodiment of the present application. The device can be configured in an electronic device and trains a text annotation model that can perform sequence annotation of multi-level labels on text by using sample text annotated with multi-level reference labels.

[0042] Among them, the electronic device can be any device with computing capabilities, such as a personal computer, mobile terminal, server, etc. The mobile terminal can be, for example, a vehicle-mounted device, a mobile phone, a tablet computer, a personal digital assistant, a wearable device, and other hardware devices with various operating systems, touch screens and / or display screens.

[0043] like Figure 1 As shown in FIG, the training method of the text annotation model includes:

[0044] Step 101: Obtain a first sample text and multi-level reference labels corresponding to each character in the first sample text.

[0045] In this application, the first sample text can be the text used in the text graph. The first sample text can be obtained when the user's text submission operation is detected, or it can be obtained from the text collection of the text graph, or it can be obtained by other means. This application does not limit this.

[0046] In this application, the first sample text can be annotated using the multi-level labels in the multi-level label set to obtain a multi-level reference label corresponding to each character in the first sample. The multi-level label set can be obtained based on a multi-level label system, which can be obtained based on a demand analysis of the text or sentence input by the user of the text map, and each level of the multi-level label system includes at least one label.

[0047] For example, in a multi-level tag system, the first-level tags include subject words, qualifiers, modifiers, topic words, grammatical words, etc. Each first-level tag has at least one second-level tag. For example, the second-level tags corresponding to topic words include scenes, scenery, and organisms. The second-level tags corresponding to qualifiers include relationships, role restrictions, and time. The second-level tags corresponding to modifiers may include styles, patterns, and artistic types. The second-level tags corresponding to topic words may include abstract concepts and colloquialisms. The second-level tags corresponding to grammatical words may include particles, affirmatives, pronouns, adverbs, and affirmatives. Some second-level tags may have at least one third-level tag. For example, the third-level tags corresponding to scenes include work, shopping, meetings, and travel. The third-level tags corresponding to organisms include animals, plants, and microorganisms.

[0048] Among them, the subject word can refer to an entity-type word, which can play a main role in the image, such as an object, a place, a building and other entities; the qualifier can be a word used to modify the "subject word", such as the sample text is glowing pebbles, then the subject word is "pebbles" and the qualifier is "glowing"; the modifier can be a word used to modify the entire text image, such as hazy beauty, sense of technology, nostalgia, etc.; the theme word can be a word used to describe the theme of the text image, such as the poem "The sun sets behind the mountains".

[0049] In this application, the number of levels of multi-level reference labels corresponding to each character in the first sample text can be the same or different, and this application does not limit this. For example, the first sample text is "luminous stone", and the corresponding annotation is: luminous (determiner_light) / (grammatical word_particle) / stone (subject word_scenery_stone).

[0050] It should be noted that the above multi-level labeling system is only an example and should not be regarded as a limitation of this application.

[0051] Step 102 : annotate the first sample text using the initial text annotation model to obtain a predicted label sequence, where the predicted label sequence includes predicted multi-level labels corresponding to each character.

[0052] In this application, the text annotation model can adopt ERNIE (Enhanced Language Representation with Informative Entities)-CRF (Conditional Random Fields), or can adopt BERT (Bidirectional Encoder Representations from Transformers, bidirectional encoding representation based on Transformer)-CRF, or can adopt LSTM (Long Short-Term Memory, long short-term memory network), etc., or other model structures can be adopted, which is not limited in this application.

[0053] In the present application, the first sample text may be input into the initial text annotation model for annotation to obtain a predicted label sequence corresponding to the first sample text, wherein the predicted label sequence includes predicted multi-level labels corresponding to each character in the first sample text.

[0054] Step 103 : training the initial text annotation model according to the difference between the predicted multi-level labels corresponding to each character and the multi-level reference labels to obtain a text annotation model.

[0055] In this application, for each character, the difference between the predicted multi-level label corresponding to the character and the multi-level reference label can be used to determine the loss corresponding to each character. Based on the loss corresponding to each character, the model loss is determined, and the parameters of the initial text annotation model are adjusted according to the model loss. The text annotation model with adjusted parameters is continued to be trained until the conditions for the end of model training are met to obtain a text annotation model.

[0056] Among them, the text annotation model can be a sequence annotation model, and the text annotation model can be used to annotate the input text of the text-graph model. The input text is annotated by using the text annotation model to obtain the annotated input text, and the annotated input text can be processed by the text-graph model to generate an image corresponding to the input text.

[0057] For example, a text annotation model is used to annotate an input text of a text-based graph model. The multi-level label of a word in the input text is "limited_time_season". Based on this label, it can be determined which season is limited. Then, when the text-based graph model generates an image based on the input text, the generated image will contain a picture of the corresponding season, thereby improving the matching degree between the generated image and the picture described in the input text.

[0058] In this application, the end condition of model training can be that the model loss is less than a preset threshold, or the number of training times reaches a preset number, etc. The end condition of model training can be set according to actual needs, and this application does not limit this.

[0059] In an embodiment of the present application, the labels corresponding to each character in the first sample text are multi-level reference labels. The first training sample is input into an initial text annotation model to obtain a predicted label sequence. Based on the difference between the predicted multi-level labels corresponding to each character in the predicted label sequence and the multi-level reference labels, a text annotation model is trained. Thus, by using the first sample text annotated with multi-level labels, a text annotation model capable of multi-level labeling can be trained. This text annotation model is used to annotate the input text of the text map with multi-level labels, thereby generating an image based on the input text annotated with multi-level labels, which can improve the matching degree between the generated image and the description of the input text.

[0060] Figure 2 A flowchart of a method for training a text annotation model provided in another embodiment of the present application.

[0061] like Figure 2 As shown, the training method of the text annotation model includes:

[0062] Step 201: Obtain a first sample text and multi-level reference labels corresponding to each character in the first sample text.

[0063] Step 202 : annotate the first sample text using the initial text annotation model to obtain a predicted label sequence, where the predicted label sequence includes predicted multi-level labels corresponding to each character.

[0064] In this application, steps 201 and 202 can refer to any implementation method in the embodiments of this application, and are not limited to this and will not be repeated.

[0065] Step 203 : Determine a first sub-loss corresponding to the character based on the difference between the predicted multi-level label corresponding to the character and the multi-level reference label.

[0066] As a possible implementation method, the predicted multi-level label and the multi-level reference label can be regarded as a label as a whole. For each character, the first sub-loss corresponding to the character can be determined based on the difference between the predicted multi-level label and the multi-level reference label corresponding to the character.

[0067] As another possible implementation, the second sub-loss corresponding to each level can be determined based on the difference between the predicted label at each level in the predicted multi-level label and the reference label at the same level in the multi-level reference label. The sum of the second sub-losses corresponding to each level in the predicted multi-level label can be used as the first sub-loss corresponding to the character. Thus, by determining the loss for each level based on the difference between the predicted multi-level label and the label at the same level in the multi-level reference label, and then determining the loss for the character based on the loss at each level, the accuracy of the loss calculation can be improved.

[0068] Step 204 : Determine a first model loss based on the first sub-loss corresponding to each character.

[0069] In this application, the sum of the first sub-losses corresponding to each character in the first sample text can be directly used as the first model loss. Alternatively, the weight of each character can be determined based on its first-level label, and the weighted sum of the first sub-losses corresponding to each character can be performed to obtain the first model loss. For example, different first-level labels have different effects on the text graph. The greater the effect, the greater the weight. For example, the weight of first-level label keywords, modifiers, etc. is higher than that of grammatical words.

[0070] Step 205: Train the initial text annotation model according to the first model loss to obtain a text annotation model.

[0071] In this application, step 205 can refer to any implementation method in the embodiments of this application, and is not limited to this and will not be repeated.

[0072] In an embodiment of the present application, the first sub-loss corresponding to each character is determined based on the difference between the predicted multi-level label and the multi-level reference label corresponding to each character, and the first model loss is determined based on the first sub-loss corresponding to each character, so that the initial text annotation model is trained based on the first model loss to obtain a text annotation model, thereby improving the accuracy of the text annotation model.

[0073] In one embodiment of the present application, a text annotation model can also be trained in the following manner: a third sub-loss can be determined based on the probability that each character output by the model belongs to a multi-level reference label, and a score corresponding to each label sequence in all possible label sequences in the first sample text is determined based on the probability that each character belongs to each multi-level label in the multi-level label set and the probability that each multi-level label is transferred to each multi-level label in the multi-level label set, and the sum of the scores of all possible label sequences is determined as the total score, and a fourth sub-loss is determined based on the ratio between the score corresponding to the predicted label sequence and the total score, and a second model loss is determined based on the third sub-loss and the fourth sub-loss, and then the parameters of the initial text annotation model are adjusted based on the second model loss, and the initial text annotation model after parameter adjustment is continued to be trained until the training end conditions are met to obtain a text annotation model.

[0074] In this application, the third sub-loss and the fourth sub-loss can be weighted and summed to obtain the second model loss. The weights can be set according to actual needs, and this application does not limit this.

[0075] Taking the ERNIE-CRF model as an example for the text annotation model, ERNIE processes the feature vector output by the first sample text and inputs it into the CRF for decoding to obtain the predicted multi-level label corresponding to the first sample text. ERNIE can replace the vector output by the last layer with the average value of the vector output by the first layer and the vector output by the last layer, that is, the average value of the vector output by the first layer and the vector output by the last layer is input into the CRF for decoding, thereby improving the accuracy of the model prediction results. The ERNIE loss (corresponding to the third sub-loss mentioned above) and the CRF loss (corresponding to the fourth sub-loss mentioned above) can be calculated, and the coefficients are adjusted in each batch of first sample texts so that the two losses in each batch of first sample texts are learned with different weights. The calculation formulas of ERNIE loss and CRF loss are as follows:

[0076]

[0077]

[0078] loss1 represents the loss of ERNIE, N represents the number of first sample texts in the current batch, for example, N can be 1, y i Indicates the position of the character in the character table, p i Indicates the probability that a character belongs to a multi-level reference label, loss2 indicates the CRF loss, S yc Represents the score of predicting multi-level label sequence, n represents the number of all possible label sequences of the first sample text, Represents the sum of the scores of all possible label sequences of the first sample text, that is, the total score.

[0079] If there are multiple Ns, the second model loss can be determined according to loss1 and the loss2 corresponding to multiple second samples respectively.

[0080] In the embodiments of the present application, by obtaining the third sub-loss according to the probability that each character belongs to the multi-level reference label, and determining the total score of the scores of all possible label sequences of the first sample text according to the probability that each character belongs to each multi-level label in the multi-level label set and the transition probability corresponding to each multi-level label, and obtaining the fourth sub-loss based on the score of the predicted label sequence and the total score, the text annotation model trained with the goal of making the predicted multi-level label be the multi-level reference label and making the proportion of the score of the true label sequence in the total score the highest improves the accuracy of the text annotation model.

[0081] Figure 3 It is a schematic flowchart of a method for training a text annotation model provided in another embodiment of the present application.

[0082] As Figure 3 shown, the method for training the text annotation model includes:

[0083] Step 301, obtain the first sample text.

[0084] In the present application, step 301 can refer to any implementation manner in the embodiments of the present application, which is not limited herein and will not be elaborated further.

[0085] Step 302, perform word segmentation on the first sample text to obtain the first word segmentation included in the first sample text.

[0086] In the present application, the first sample text can be segmented according to a preset rule to obtain the first word segmentation included in the first sample text. Among them, for example, the word segmentation with the predicted rule of being a noun and the adjacent character with the predicted rule of being a verb can be not segmented, for example, "hold by hand" does not need to be segmented into "hand / hold", and "hold by hand" is directly used as a word segmentation.

[0087] Alternatively, a pre-trained word segmentation model can also be used to perform word segmentation on the first sample text to obtain the first word segmentation included in the first sample text.

[0088] Step 303, match the first word segmentation with the entities and the relationships between the entities in the preset graph to determine the multi-level reference label corresponding to the first word segmentation from the multi-level label set.

[0089] In the present application, the multi-level reference label corresponding to the first word segmentation can be determined by using the graph features of the preset graph. Among them, the preset graph can be a knowledge graph, a concept graph, etc., or other graphs can also be used. The used prediction graph can be one or more, which is not limited in the present application.

[0090] As an implementation method, the first segmentation can be matched with entities and / or relationships between entities in a preset graph. If the first target segmentation in the first segmentation matches the first entity in the preset graph, it means that the first target segmentation represents an entity, but the entities have different roles in the text graph. The attributes of the first entity and / or the relationship between the first entity and adjacent entities can be further matched with the first candidate multi-level label to determine the multi-level reference label corresponding to the first target segmentation. The first candidate multi-level label can refer to a multi-level label in a multi-level label set whose first-level label is the first preset label, and the first predicted label can be a main word, a modifier, etc.

[0091] For example, the word "camera" in a first sample text is matched with the entity "camera" in the knowledge graph. The attributes of "camera" in the knowledge graph, such as being used for videography, artistic type photography, etc., and the relationship "hand-held" between the entity "camera" and the adjacent entity "A" can be further matched with multi-level labels whose first-level labels are subject words and modifiers. It can be determined that the multi-level label corresponding to the word "camera" can be modifier-art type-videography, where A represents a person's name.

[0092] If the second target segmentation in the first segmentation does not match any entity in the knowledge graph, the second target segmentation can be matched with the relationship between entities in the preset graph. If the second target segmentation matches the relationship between two entities in the preset graph, the attributes of the relationship between the two entities can be matched with the second candidate multi-level label. Based on the matching results, the multi-level reference label corresponding to the second target segmentation is determined. The second candidate multi-level label can be a multi-level label in the multi-level label set whose first-level label is the second preset label. The second preset label can be a qualifier, etc.

[0093] For example, the word "hand holding" in a first sample text matches the relationship "hand support" between two entities in the knowledge graph. Matching can be performed based on the attributes of "hand support", such as a person's action posture, to determine the multi-level reference label corresponding to "hand holding" as qualifier-role-action posture.

[0094] If the third target participle in the first participle does not match the entity in the preset graph and the relationship between the entities, the third target participle can be matched with the attributes of the entity in the sentence to which the third target participle belongs to determine the multi-level reference label corresponding to the third target participle.

[0095] For example, a sentence in a first sample text is "some shining stones". The word "some" does not match the entities and the relationship between entities in the knowledge graph. The word can be matched with the attributes of the entity "stone" in the sentence, such as the number and weight of the stones, and the multi-level reference label corresponding to the word can be determined to be the qualifier-quantity.

[0096] Thus, by matching the first segmented word with the entities and the relationships between the entities in the preset atlas, different methods are adopted to determine the multi-level reference labels of the first segmented word according to different matching results, thereby achieving the annotation of the first sample text and improving the accuracy of the annotation result of the first sample text.

[0097] Step 304: Annotate the characters in the first segmented word according to the multi-level reference labels corresponding to the first segmented word, so as to obtain the multi-level reference labels corresponding to the characters.

[0098] In this application, if the first segmented word contains multiple characters, the multi-level reference labels corresponding to the first segmented word can be used as the multi-level reference labels corresponding to each character included in the first segmented word, and then the characters are annotated, so that the multi-level reference labels corresponding to the characters can be obtained.

[0099] For example, the multi-level reference labels corresponding to each first segmented word in a certain first sample text are as follows: some (determiner_quantity) / luminous (determiner_light) / of (grammatical_word_auxiliary) / stone (subject_word_scenery_stone). Then the multi-level reference labels corresponding to the characters "一" and "些" are both determiner_quantity, the multi-level reference labels corresponding to the characters "发" and "光" are both determiner_light, and the multi-level reference labels corresponding to the characters "石" and "头" are both subject_word_scenery_stone.

[0100] In this application, for some special characters, the multi-level reference labels corresponding to these characters can be determined according to preset rules. For example, the multi-level reference label corresponding to a punctuation mark is grammatical_word-w, and the multi-level reference label corresponding to the character "的" after a modifier is grammatical_word-auxiliary, etc.

[0101] Step 305: Use the initial text annotation model to annotate the first sample text to obtain a predicted label sequence, and the predicted label sequence includes the predicted multi-level labels corresponding to each character respectively.

[0102] Step 306: Train the initial text annotation model according to the differences between the predicted multi-level labels corresponding to each character respectively and the multi-level reference labels, so as to obtain a text annotation model.

[0103] In this application, Steps 305-Step 306 can refer to any implementation manner in the embodiments of this application, which is not limited herein and will not be elaborated further.

[0104] In an embodiment of the present application, a first sample text can be segmented to obtain a first segmented word contained in the first sample text. By matching the first segmented word with entities and the relationships between entities in a preset graph, a multi-level reference label corresponding to the first segmented word is determined from a multi-level label set. Then, based on the multi-level reference label corresponding to the first segmented word, the characters in the first segmented word are annotated to obtain the multi-level reference labels corresponding to the characters. Thus, by using a preset knowledge graph to determine the multi-level reference labels corresponding to each segmented word in the first sample, and then determining the multi-level reference labels corresponding to the characters, automatic annotation of each character in the first sample text is achieved, thereby improving the accuracy of the annotation results.

[0105] In one embodiment of the present application, the multi-level reference label corresponding to each character in the first sample text may also be determined according to the part of speech of each first word segment in the first sample text.

[0106] In this application, the first sample text can be segmented to obtain the first participle contained in the first sample text, and the part of speech of the participle can be determined. According to the part of speech of the first participle and the adjacent participles of the first participle, the multi-level reference label corresponding to the first participle is determined from the multi-level label set, and according to the multi-level reference label corresponding to the first participle, the characters in the first participle are marked to obtain the multi-level reference labels corresponding to the characters.

[0107] The word segmentation processing method may refer to the above embodiment, and the part-of-speech tagging model may be used to perform part-of-speech tagging on the segmented words to determine the part-of-speech of the segmented words.

[0108] When determining the multi-level reference label corresponding to a first participle based on the part of speech of the first participle and the part of speech of the adjacent participles, the first-level candidate label corresponding to the first participle can be determined based on a preset mapping relationship between the part of speech and the first-level label, and the first reference label can be determined from the first-level candidate labels based on the part of speech of the adjacent participles of the first participle. The first participle is then matched with the labels of the remaining levels of the third candidate multi-level label except the first-level label to determine the reference labels of the remaining levels. The first reference label and the reference labels of the remaining levels are combined to determine the multi-level reference label corresponding to the first participle. The third candidate multi-level label can be a multi-level label in the multi-level label set whose first-level label is the first reference label.

[0109] Therefore, the first-level candidate label can be determined according to the mapping relationship, and then the first reference label can be determined from the first-level candidate labels according to the parts of speech of adjacent participles. Then, according to the matching situation between the first participle and the labels of the remaining levels in the third candidate multi-level label, the reference labels of the remaining levels can be determined, thereby determining the multi-level reference label corresponding to the first participle. This realizes the determination of the multi-level reference label corresponding to the participle according to the part of speech of the participle, and enriches the method of determining the multi-level reference label of the participle.

[0110] As an implementation method, the sentence component of the first participle in the sentence to which it belongs can be determined based on the part of speech of the first participle's adjacent participles, and a first reference label that matches the sentence component can be determined from the first-level candidate labels based on the sentence component of the first participle in the sentence to which it belongs. Thus, the sentence component of the first participle in the sentence to which it belongs can be determined based on the part of speech of the first participle's adjacent participles, and the first-level reference label of the first participle can be determined based on the sentence component of the first participle in the sentence to which it belongs, thus achieving the determination of the first-level reference label of the first participle based on the part of speech of the first participle's adjacent participles.

[0111] When determining the reference labels of the remaining levels, the semantic matching degree between the first word segmentation and the labels of the remaining levels in the third candidate multi-level label can be calculated, and the labels with semantic matching degree greater than the preset matching degree can be used as reference labels of the remaining levels.

[0112] For example, the mapping relationship between parts of speech and first-level labels can be that nouns correspond to main words and modifiers, adjectives correspond to modifiers, verbs correspond to qualifiers, etc. A first sample text includes the sentence "some glowing stones". According to this mapping relationship, the first-level candidate labels of the segmentation word "stone" can be determined as the main word and modifier. According to the adjacent segmentations "luminous" and "of" of "stone", the segmentation word "stone" can be the subject in the sentence, and "some glowing" is the attributive. The first reference label of the segmentation word "stone" can be determined to be the main word, and the semantic matching degree of "stone" with the third candidate multi-level labels other than the main word in the main word-scene, main word-scenery, main word-scenery-stone, main word-organism, etc. is calculated. The second-level label can be determined to be scenery and the third-level label can be determined to be stone. Therefore, the multi-level reference label corresponding to the segmentation word "stone" can be determined to be main word-scenery-stone.

[0113] In an embodiment of the present application, a multi-level reference label corresponding to the first participle can be determined from a multi-level label set based on the part of speech of the first participle and the part of speech of adjacent participles in the first sample text, and the characters in the first participle are labeled based on the multi-level reference label corresponding to the first participle to obtain the multi-level reference labels corresponding to the characters. Thus, by utilizing the part of speech of the first participle and the adjacent participles, the multi-level reference label corresponding to the first participle is determined, and then the multi-level reference label corresponding to the characters is determined, thereby achieving automatic labeling of each character in the first sample text and improving the accuracy of the labeling results.

[0114] Figure 4 A flowchart of a method for training a text annotation model provided in another embodiment of the present application.

[0115] like Figure 4 As shown, the training method of the text annotation model includes:

[0116] Step 401: Obtain a second sample text.

[0117] In the present application, the second sample text may be text used for a text map. The method for obtaining the second sample text is similar to the method for obtaining the first sample text, so it will not be described in detail here.

[0118] Step 402 : Perform a text-based graph requirement analysis on the second sample text to obtain a multi-level tag system, where each level of the multi-level tag system includes at least one tag.

[0119] Among them, the multi-level label system can include multiple levels of labels, each level can include at least one label, and the subordinate levels contained in the same level can be the same or different, which is not limited in this application.

[0120] For example, the first-level label is a qualifier, and the second-level labels included in the qualifier are: background limitation, relationship, light, and role limitation. The second-level label relationship also includes the third-level label processing action, and the second-level label role limitation also includes the third-level labels action posture, age, emotion, etc., but the second-level label light does not have a third-level label.

[0121] In this application, the second sample text can be segmented to obtain the second segmented words contained in the second sample text, and the second segmented words can be analyzed for text-based image requirements to obtain information about the requirements of the text-based image scenario. Based on this information, a multi-level labeling system can be obtained. Thus, by mining and analyzing the user's text-based image requirements, a multi-level labeling system can be obtained, so that the multi-level labels in the multi-level labeling system can reflect the text-based image requirements.

[0122] The word segmentation processing method for the second sample text can refer to the word segmentation processing method for the first sample text in the above embodiment, so it will not be described in detail here.

[0123] As an implementation method, the second participle can be subjected to syntactic analysis and semantic recognition to determine the sentence components of the second participle in the sentence to which it belongs in the second sample text and the semantic information of the second participle. Based on the sentence components and semantic information, demand information can be determined, and then a multi-level label system can be obtained based on the demand information. In this way, by determining demand information based on the sentence components of the second participle in the sentence to which it belongs and the semantic information of the participle, a multi-level label system can be obtained based on the demand of the text graph.

[0124] Among them, the sentence component of the second participle in the sentence to which it belongs can be determined based on the part of speech of the second participle and the part of speech of the adjacent participles of the second participle, or the sentence component of the second participle in the sentence to which it belongs can be determined by using a model, and the semantics of the second participle can be identified by using a semantic recognition model to obtain the semantic information of the second participle.

[0125] For example, a second sample text is "B holding an apple", where the participle "B" is the subject, the participle "holding" is the predicate, and the participle "apple" is the object. Taking the participle "holding" as an example, "holding" is the predicate, and the semantic information is the action. It can be determined that there is a need for action restriction on B, and thus it can be determined that the first-level label has a qualifier, the second-level label included in the qualifier can have a role restriction, and the third-level label included in the role restriction has an action restriction.

[0126] Step 403: Acquire a multi-level tag set according to the multi-level tag system.

[0127] In this application, the first-level label in the multi-level label system can be combined with the second-level label under the first-level label to obtain a first label. If the second-level label in the first label has a third-level label, the first label can be combined with the third label to obtain a second label, until it is combined with the last-level label to obtain the Nth label, and a multi-label set is obtained based on the first label, the second label, and the Nth label. Wherein, N can be an integer greater than 2. Thus, based on the multi-level label system, a multi-level label set containing multi-level labels can be obtained, and the multi-level label set can be used for annotation to realize the annotation of the input text of the Wensheng diagram.

[0128] As an example, if the third-level tag in the second tag has a fourth-level tag, the second tag and the fourth-level tag can be combined to obtain a third tag. If the fourth-level tag in the third tag has a fifth-level tag, the third tag and the fifth-level tag can be combined to obtain a fourth tag. Based on the first tag, the second tag, the third tag, and the fourth tag, a multi-level tag set can be obtained. The multi-level tag set includes the first tag, the second tag, the third tag, and the fourth tag.

[0129] For example, in a multi-level labeling system, the first-level labels include subject words, qualifiers, modifiers, theme words, grammatical words, etc. The second-level labels corresponding to theme words include scenes, scenery, organisms, etc. The second-level labels corresponding to qualifiers include relationships, role limitations, time, etc. The second-level labels corresponding to modifiers can include styles, patterns, art types, etc. The second-level labels corresponding to theme words can include abstract concepts, colloquialisms, etc. The second-level labels corresponding to grammatical words can include auxiliary words, affirmative words, pronouns, adverbs, affirmative words, etc. The third-level labels corresponding to scenes include work, shopping, meetings, travel, etc. The third-level labels corresponding to organisms include animals, plants, etc. Objects, microorganisms, etc., then the first label can include subject word-scene, subject word-scenery, subject word-organism, qualifier-relationship, modifier-style, modifier-mode, modifier-art type, qualifier-role limitation, qualifier-time, subject word-abstract concept, subject word-verb, grammatical word-particle, grammatical word-affirmative word, grammatical word-pronoun, grammatical word-adverb, grammatical word-affirmative word, and the second label can include subject word-scene-work, subject word-scene-shopping, subject word-scene-meeting, subject word-scene-travel, etc. A multi-level label set can be obtained based on the first label and the second label.

[0130] It should be noted that the above-mentioned multi-level label system and multi-level label set are only examples and can be determined according to actual needs. This application does not limit this.

[0131] In related technologies, word segmentation models, part-of-speech tagging models, and proper name tagging models can be used to annotate text. However, these models cannot effectively solve text comprehension-related problems of text-generated graphs, such as controllable generation problems and text polishing problems.

[0132] In the embodiment of the present application, a multi-level label system can be obtained by performing a text graph demand analysis on the second sample text, and a multi-level label set can be obtained based on the multi-level label system. The multi-level labels in the multi-level label set can be used to annotate the first sample text, and the text annotation model can be trained using the annotated first sample text to obtain the text annotation model, so that the input text of the text graph can be annotated using the text annotation model. It can be achieved that the annotation is performed according to the specific needs of the text graph, and the user's needs are understood, so that the input text annotated by the text annotation model is used to generate the text graph, thereby solving the problem of controllable generation of the text graph, making the text graph controllable, and improving the matching degree between the generated image and the description of the input text.

[0133] If the annotation results of the input text reveal that some content is missing, you can supplement the missing content and polish the input text, thereby improving the effect of the text map. For example, based on the annotation results of the text annotation model, you can determine whether the input text already has a subject word and modifiers. If there are no subject words and modifiers, you can add some subject words and modifiers, and use the supplemented input text to create a text map.

[0134] Figure 5 A flowchart of a text-generated diagram method provided in one embodiment of the present application is shown.

[0135] like Figure 5 As shown, the text-generated graph method includes:

[0136] Step 501: Obtain target text.

[0137] In this application, the target text can be the text or sentence input by the user of the text image. If the user inputs voice, the collected voice can be recognized and converted into text.

[0138] Step 502: annotate the target text using a text annotation model to obtain a sequence annotation result corresponding to the target text.

[0139] The text annotation model can be trained using the above-mentioned text annotation model training method, and the text annotation model can perform sequence annotation on the text.

[0140] In this application, the target text can be input into the text annotation model for annotation, and the multi-level labels corresponding to each character in the target text output by the text annotation model can be obtained. According to the multi-level labels corresponding to the characters belonging to the same word, the multi-level labels corresponding to each word in the target text can be determined, thereby obtaining the sequence annotation results corresponding to the target text.

[0141] For example, the target text is "There are some glowing stones and colorful pebbles on the beach, pink background, cyberpunk style", and the annotation result of the target text is "on the beach (grammatical word_position direction) / there are (grammatical word_affirmative word) / some (limited_quantity) / glowing (limited_light) / of (grammatical word_particle) / stones (subject_scenery_stone) / , (grammatical word_w) / and (grammatical word_affirmative word) / colorful (modifier_mode_tone_colorful) / of (grammatical word_particle) / pebbles (subject_scenery_stone) / , (grammatical word_w) / pink background (modifier_mode_tone_background tone) / , (grammatical word_w) / cyberpunk style (modifier_style_genre)".

[0142] Step 503: Input the target text and the sequence annotation results into the text-graph model for processing to generate an image corresponding to the target text.

[0143] In this application, the target text and the sequence annotation results of the target text can be input into the text-based graph model for processing to generate an image corresponding to the target text, thereby realizing image generation based on text.

[0144] Furthermore, in order to improve the effect of the text graph, if it is determined based on the annotation results of the target text that the target text is missing some content, the target text can be supplemented and then input into the text graph model for processing; or if the annotation results of some word segments of the target text are inaccurate, the annotation results can be corrected, and then the target text and the corrected annotation results can be input into the text graph model for processing to obtain the corresponding image.

[0145] In an embodiment of the present application, the target text can be annotated with multi-level labels by using a text annotation model, and then the target text annotated with multi-level labels can be processed and an image generated by using a text-based graph model, which can improve the matching degree between the generated image and the description of the target text.

[0146] In an embodiment of the present application, a text annotation model can be trained using sample text annotated with a multi-level label system obtained based on mining and analysis of text graph needs. The text annotation model can be used to perform multi-level label annotating on the target text, and the target text can be annotated according to the needs of text graph users. The text graph model can then be used to process the target text annotated with multi-level labels to generate an image, which can improve the matching degree between the generated image and the description of the target text.

[0147] In order to implement the above embodiment, the embodiment of the present application also proposes a training device for a text annotation model. Figure 6 A schematic diagram of the structure of a training device for a text annotation model provided in one embodiment of the present application.

[0148] like Figure 6 As shown, the training device 600 of the text annotation model includes:

[0149] A first acquisition module 610 is configured to acquire a first sample text and a multi-level reference label corresponding to each character in the first sample text;

[0150] A second acquisition module 620 is configured to annotate the first sample text using the initial text annotation model to obtain a predicted label sequence, where the predicted label sequence includes predicted multi-level labels corresponding to each character;

[0151] A training module 630 is configured to train the initial text annotation model based on the difference between the predicted multi-level labels corresponding to each character and the multi-level reference labels to obtain a text annotation model;

[0152] The text annotation model is used to annotate the input text of the text-to-graph model, and the text-to-graph model is used to process the annotated input text to generate an image corresponding to the input text.

[0153] Optionally, the training module 630 is configured to:

[0154] Determine the first sub-loss corresponding to the character based on the difference between the predicted multi-level label corresponding to the character and the multi-level reference label;

[0155] Determine the first model loss according to the first sub-loss corresponding to each character;

[0156] The initial text annotation model is trained according to the first model loss to obtain a text annotation model.

[0157] Optionally, the training module 630 is configured to:

[0158] Determine the second sub-loss corresponding to each level according to the difference between the predicted label of each level in the predicted multi-level label and the reference label of the same level in the multi-level reference label;

[0159] The first sub-loss is determined according to the second sub-loss corresponding to each level in the predicted multi-level label.

[0160] Optionally, the training module 630 is configured to:

[0161] Determine the third sub-loss based on the probability of each character belonging to the multi-level reference label;

[0162] Determine a score corresponding to each label sequence in all possible label sequences of the first sample text according to the probability that each character belongs to each multi-level label in the multi-level label set and the probability that each multi-level label is transferred to each multi-level label in the multi-level label set;

[0163] Determine the total score based on the sum of the scores of all possible tag sequences;

[0164] Determine the fourth sub-loss based on the score corresponding to the predicted label sequence and the total score;

[0165] Determine the second model loss according to the third sub-loss and the fourth sub-loss;

[0166] The initial text annotation model is trained according to the second model loss to obtain a text annotation model.

[0167] Optionally, the first acquisition module 610 is configured to:

[0168] Get the first sample text;

[0169] Performing word segmentation processing on the first sample text to obtain a first word segmentation included in the first sample text;

[0170] Match the first segmented word with entities and the relationships between entities in the preset graph, and determine the multi-level reference label corresponding to the first segmented word from the multi-level label set;

[0171] According to the multi-level reference labels corresponding to the first segmented word, the characters in the first segmented word are labeled to obtain the multi-level reference labels corresponding to the characters.

[0172] Optionally, the first acquisition module 610 is configured to:

[0173] When the first target participle in the first participle matches the first entity in the preset graph, matching the attributes of the first entity and / or the relationship between the first entity and the adjacent entities with the first candidate multi-level label to determine the multi-level reference label corresponding to the first target participle;

[0174] When the second target segmentation in the first segmentation matches the relationship between two entities in the preset graph, the attributes of the relationship between any two entities are matched with the second candidate multi-level labels to determine the multi-level reference label corresponding to the second target segmentation;

[0175] If the third target participle in the first participle does not match any entity or relationship between entities in the preset graph, the third target participle is matched with the attributes of the entity in the sentence to which the third target participle belongs, and a multi-level reference label corresponding to the third target participle is determined;

[0176] The first candidate multi-level tag refers to a multi-level tag whose first level tag is the first preset tag in the multi-level tag set, and the second candidate multi-level tag refers to a multi-level tag whose first level tag is the second preset tag in the multi-level tag set.

[0177] Optionally, the first acquisition module 610 is configured to:

[0178] Get the first sample text;

[0179] Performing word segmentation processing on the first sample text to obtain a first word segmentation contained in the first sample text and determining the part of speech of the word segmentation;

[0180] According to the part of speech of the first participle and the parts of speech of adjacent participles of the first participle, a multi-level reference label corresponding to the first participle is determined from the multi-level label set;

[0181] According to the multi-level reference labels corresponding to the first segmented word, the characters in the first segmented word are labeled to obtain the multi-level reference labels corresponding to the characters.

[0182] Optionally, the first acquisition module 610 is configured to:

[0183] According to the mapping relationship between part of speech and first-level labels, determine the first-level candidate label corresponding to the first participle;

[0184] Determine a first reference label from the first-level candidate labels according to the parts of speech of the adjacent participles of the first participle;

[0185] Match the first word segment with the labels of the remaining levels except the first-level label in the third candidate multi-level label to determine the reference labels of the remaining levels, where the third candidate multi-level label refers to the multi-level label in the multi-level label set whose first-level label is the first reference label;

[0186] A multi-level reference label is determined based on the first reference label and the reference labels of the remaining levels.

[0187] Optionally, the first acquisition module 610 is configured to:

[0188] Determine the sentence component of the first participle in the sentence to which it belongs according to the part of speech of the adjacent participles of the first participle;

[0189] A first reference tag matching the sentence component is determined from the first-level candidate tags.

[0190] Optionally, the device may further include:

[0191] A second acquisition module, used to acquire a second sample text;

[0192] A third acquisition module is configured to perform a text-image demand analysis on the second sample text to obtain a multi-level label system, where each level of the multi-level label system includes at least one label;

[0193] The fourth acquisition module is used to acquire a multi-level tag set according to the multi-level tag system.

[0194] Optionally, the fourth acquisition module is configured to:

[0195] Combine the first-level label in the multi-level label system with the second-level label under the first-level label to obtain a first label;

[0196] In the case where the second-level tag in the first tag has a third-level tag, the first tag is combined with the third-level tag to obtain the second tag, and the combination is continued until it is combined with the last-level tag to obtain the Nth tag, where N is an integer greater than 2;

[0197] A multi-level tag set is obtained based on the first tag, the second tag, and so on to the Nth tag.

[0198] Optionally, the third acquisition module is used to:

[0199] Performing word segmentation processing on the second sample text to obtain a second word segmentation included in the second sample text;

[0200] Conduct text-image demand analysis on the second participle to obtain demand information for the text-image scenario;

[0201] Obtain a multi-level labeling system based on demand information.

[0202] Optionally, the third acquisition module is used to:

[0203] Performing syntactic analysis and semantic recognition on the second participle to determine the sentence component of the second participle in the sentence to which it belongs and the semantic information of the second participle;

[0204] Determine the required information based on sentence components and semantic information;

[0205] Obtain a multi-level labeling system based on demand information.

[0206] It should be noted that the explanation of the embodiment of the training method of the text annotation model mentioned above is also applicable to the training device of the text annotation model of this embodiment, so it will not be repeated here.

[0207] In an embodiment of the present application, the labels corresponding to each character in the first sample text are multi-level reference labels. The first training sample is input into an initial text annotation model to obtain a predicted label sequence. Based on the difference between the predicted multi-level labels corresponding to each character in the predicted label sequence and the multi-level reference labels, a text annotation model is trained. Thus, by using the first sample text annotated with multi-level labels, a text annotation model capable of multi-level labeling can be trained. This text annotation model is used to annotate the input text of the text map with multi-level labels, thereby generating an image based on the input text annotated with multi-level labels, which can improve the matching degree between the generated image and the description of the input text.

[0208] In order to implement the above embodiment, the embodiment of the present application further proposes a cultural image device. Figure 7 This is a structural diagram of a text-generating image device provided in one embodiment of the present application.

[0209] like Figure 7 As shown, the text image device 700 includes:

[0210] A first acquisition module 710 is used to acquire a target text;

[0211] A second acquisition module 720 is configured to annotate the target text using a text annotation model to obtain a sequence annotation result corresponding to the target text, wherein the text annotation model is trained using the method described in the above embodiment;

[0212] The generating module 730 is configured to input the target text and the sequence annotation result into a text-graph model for processing to generate an image corresponding to the target text.

[0213] It should be noted that the explanations of the aforementioned embodiment of the Wenshengtu method are also applicable to the Wenshengtu device of this embodiment, and therefore will not be repeated here.

[0214] In an embodiment of the present application, the target text can be annotated with multi-level labels by using a text annotation model, and then the target text annotated with multi-level labels can be processed and an image generated by using a text-based graph model, which can improve the matching degree between the generated image and the description of the target text.

[0215] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0216] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0217] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 802 or a computer program loaded from a storage unit 808 into a RAM (Random Access Memory) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An I / O (Input / Output) interface 805 is also connected to the bus 804.

[0218] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0219] The computing unit 801 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a CPU (Central Processing Unit), a GPU (Graphic Processing Unit), various specialized AI (Artificial Intelligence) computing chips, various computing units that run machine learning model algorithms, a DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the training method for the text annotation model. For example, in some embodiments, the training method for the text annotation model can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the training method for the text annotation model described above can be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the training method of the text annotation model in any other appropriate manner (for example, by means of firmware).

[0220] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System on Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special-purpose or general-purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0221] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0222] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0223] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0224] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.

[0225] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. The client-server relationship is established by computer programs running on the respective computers and establishing a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and poor scalability of traditional physical hosts and VPS services. The server may also be a server in a distributed system or a server integrated with blockchain.

[0226] It should be noted that the electronic device used to implement the Wenshengtu method of the embodiment of the present application has a structure similar to that of the above-mentioned electronic device, so it will not be described in detail here.

[0227] According to an embodiment of the present application, the present application also provides a computer program product, which, when the instruction processor in the computer program product is executed, executes the text annotation model training method or text-to-graph method proposed in the above embodiments of the present application.

[0228] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0229] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A training method for a text annotation model, comprising: Obtaining a first sample text and multi-level reference labels corresponding to each character in the first sample text; Annotating the first sample text using an initial text annotation model to obtain a predicted label sequence, wherein the predicted label sequence includes predicted multi-level labels corresponding to each of the characters; Training the initial text annotation model according to the difference between the predicted multi-level labels corresponding to each of the characters and the multi-level reference labels to obtain a text annotation model; The text annotation model is used to annotate the input text of the text-generated graph model, and the text-generated graph model is used to process the annotated input text to generate an image corresponding to the input text; The obtaining of the first sample text and the multi-level reference labels corresponding to the characters in the first sample text includes: Obtaining the first sample text; Performing word segmentation processing on the first sample text to obtain a first word segmentation included in the first sample text; Matching the first segmented word with entities and relationships between entities in a preset graph, and determining a multi-level reference label corresponding to the first segmented word from a multi-level label set; Labeling characters in the first segmented word according to the multi-level reference labels corresponding to the first segmented word to obtain the multi-level reference labels corresponding to the characters; Matching the first segmented word with entities in a preset graph and the relationships between the entities to determine a multi-level reference label corresponding to the first segmented word includes: When a first target participle in the first participle matches a first entity in the preset graph, matching attributes of the first entity and / or a relationship between the first entity and adjacent entities with a first candidate multi-level label to determine a multi-level reference label corresponding to the first target participle; When a second target word in the first word segmentation matches a relationship between two entities in the preset graph, matching the attributes of the relationship between any two entities with a second candidate multi-level label to determine a multi-level reference label corresponding to the second target word segmentation; If the third target participle in the first participle does not match any entity in the preset graph or any relationship between entities, matching the third target participle with attributes of the entity in the sentence to which the third target participle belongs, and determining a multi-level reference label corresponding to the third target participle; The first candidate multi-level tag refers to a multi-level tag in the multi-level tag set whose first-level tag is a first preset tag, and the second candidate multi-level tag refers to a multi-level tag in the multi-level tag set whose first-level tag is a second preset tag.

2. The method according to claim 1, wherein The initial text annotation model is trained according to the difference between the predicted multi-level labels corresponding to each of the characters and the multi-level reference labels to obtain a text annotation model, including: Determining a first sub-loss corresponding to the character according to a difference between the predicted multi-level label corresponding to the character and the multi-level reference label; Determining a first model loss according to the first sub-losses corresponding to the characters; The initial text annotation model is trained according to the first model loss to obtain the text annotation model.

3. The method according to claim 2, wherein: The determining, according to a difference between the predicted multi-level label corresponding to the character and the multi-level reference label, a first sub-loss corresponding to the character, comprises: Determining a second sub-loss corresponding to each level according to a difference between a predicted label of each level in the predicted multi-level labels and a reference label of the same level in the multi-level reference labels; The first sub-loss is determined according to the second sub-losses corresponding to each level in the predicted multi-level label.

4. The method according to claim 1, wherein The initial text annotation model is trained according to the difference between the predicted multi-level labels corresponding to each of the characters and the multi-level reference labels to obtain a text annotation model, including: determining a third sub-loss according to a probability that each of the characters belongs to the multi-level reference labels; Determine a score corresponding to each label sequence in all possible label sequences of the first sample text according to the probability that each character belongs to each multi-level label in the multi-level label set and the probability that each multi-level label is transferred to each multi-level label in the multi-level label set; Determine the total score based on the sum of the scores of all possible tag sequences; Determining a fourth sub-loss according to the score corresponding to the predicted label sequence and the total score; determining a second model loss according to the third sub-loss and the fourth sub-loss; The initial text annotation model is trained according to the second model loss to obtain the text annotation model.

5. The method according to claim 1, wherein The obtaining of the first sample text and the multi-level reference labels corresponding to the characters in the first sample text includes: Obtaining the first sample text; Performing word segmentation processing on the first sample text to obtain a first word segmentation included in the first sample text, and determining the part of speech of the first word segmentation; Determining, from a multi-level tag set, a multi-level reference tag corresponding to the first participle according to the part of speech of the first participle and the parts of speech of adjacent participles of the first participle; According to the multi-level reference labels corresponding to the first word segmentation, characters in the first word segmentation are labeled to obtain the multi-level reference labels corresponding to the characters.

6. The method according to claim 5, wherein: The determining the multi-level reference labels according to the part of speech of the first participle and the part of speech of adjacent participles of the first participle includes: Determine the first-level candidate label corresponding to the first word segmentation according to the mapping relationship between the part of speech and the first-level label; Determining a first reference tag from the first-level candidate tags based on the parts of speech of adjacent participles of the first participle; Matching the first word segmentation with labels of the remaining levels except the first-level label in a third candidate multi-level label to determine reference labels of the remaining levels, wherein the third candidate multi-level label refers to a multi-level label in the multi-level label set whose first-level label is the first reference label; The multi-level reference labels are determined according to the first reference label and the reference labels of the remaining levels.

7. The method according to claim 6, wherein: The step of determining a first reference tag from the first-level candidate tags based on the parts of speech of adjacent participles of the first participle includes: determining, based on the parts of speech of adjacent participles of the first participle, the sentence component of the first participle in the sentence to which it belongs; The first reference tag matching the sentence component is determined from the first-level candidate tags.

8. The method according to any one of claims 4 to 7, further comprising: Get the second sample text; Performing a text-based graph demand analysis on the second sample text to obtain a multi-level label system, where each level of the multi-level label system includes at least one label; According to the multi-level label system, the multi-level label set is obtained.

9. The method of claim 8, wherein: The step of obtaining the multi-level tag set according to the multi-level tag system includes: Combining a primary tag in the multi-level tag system with a secondary tag under the primary tag to obtain a first tag; In the case where the second-level tag in the first tag has a third-level tag, the first tag is combined with the third-level tag to obtain a second tag, and the combination is continued until the combination is combined with the last-level tag to obtain an Nth tag, where N is an integer greater than 2; A multi-level tag set is acquired according to the first tag, the second tag, and up to the Nth tag.

10. The method of claim 8, wherein: The performing text-based graph demand analysis on the second sample text to obtain a multi-level labeling system includes: Performing word segmentation processing on the second sample text to obtain second word segments contained in the second sample text; Performing a text-image demand analysis on the second participle to obtain demand information of the text-image scenario; The multi-level label system is acquired according to the demand information.

11. The method according to claim 10, wherein: The performing text-image demand analysis on the second participle to obtain demand information of the text-image scenario includes: performing syntactic analysis and semantic recognition on the second participle respectively to determine the sentence component of the second participle in the sentence to which it belongs and the semantic information of the second participle; determining the demand information according to the sentence components and the semantic information; The multi-level label system is acquired according to the demand information.

12. A method for generating a text map, comprising: Get the target text; Annotating the target text using a text annotation model to obtain a sequence annotation result corresponding to the target text, wherein the text annotation model is trained using the method according to any one of claims 1 to 11; The target text and the sequence annotation result are input into a text-based graph model for processing to generate an image corresponding to the target text.

13. A training device for a text annotation model, comprising: A first acquisition module is configured to acquire a first sample text and a multi-level reference label corresponding to each character in the first sample text; A second acquisition module is configured to annotate the first sample text using an initial text annotation model to obtain a predicted label sequence, wherein the predicted label sequence includes predicted multi-level labels corresponding to each of the characters; A training module, configured to train the initial text annotation model according to the difference between the predicted multi-level labels corresponding to the characters and the multi-level reference labels, so as to obtain a text annotation model; The text annotation model is used to annotate the input text of the text-generated graph model, and the text-generated graph model is used to process the annotated input text to generate an image corresponding to the input text; The first acquisition module is configured to: Obtaining the first sample text; Performing word segmentation processing on the first sample text to obtain a first word segmentation included in the first sample text; Matching the first segmented word with entities and relationships between entities in a preset graph, and determining a multi-level reference label corresponding to the first segmented word from a multi-level label set; Labeling characters in the first segmented word according to the multi-level reference labels corresponding to the first segmented word to obtain the multi-level reference labels corresponding to the characters; The first acquisition module is further configured to: When a first target participle in the first participle matches a first entity in the preset graph, matching attributes of the first entity and / or a relationship between the first entity and adjacent entities with a first candidate multi-level label to determine a multi-level reference label corresponding to the first target participle; When a second target word in the first word segmentation matches a relationship between two entities in the preset graph, matching the attributes of the relationship between any two entities with a second candidate multi-level label to determine a multi-level reference label corresponding to the second target word segmentation; If the third target participle in the first participle does not match any entity in the preset graph or any relationship between entities, matching the third target participle with attributes of the entity in the sentence to which the third target participle belongs, and determining a multi-level reference label corresponding to the third target participle; The first candidate multi-level tag refers to a multi-level tag in the multi-level tag set whose first-level tag is a first preset tag, and the second candidate multi-level tag refers to a multi-level tag in the multi-level tag set whose first-level tag is a second preset tag.

14. The apparatus of claim 13, wherein: The training module is used to: Determining a first sub-loss corresponding to the character according to a difference between the predicted multi-level label corresponding to the character and the multi-level reference label; Determining a first model loss according to the first sub-losses corresponding to the characters; The initial text annotation model is trained according to the first model loss to obtain the text annotation model.

15. The apparatus of claim 14, wherein: The training module is used to: Determining a second sub-loss corresponding to each level according to a difference between a predicted label of each level in the predicted multi-level labels and a reference label of the same level in the multi-level reference labels; The first sub-loss is determined according to the second sub-losses corresponding to each level in the predicted multi-level label.

16. The apparatus of claim 13, wherein: The training module is used to: determining a third sub-loss according to a probability that each of the characters belongs to the multi-level reference labels; Determine a score corresponding to each label sequence in all possible label sequences of the first sample text according to the probability that each character belongs to each multi-level label in the multi-level label set and the probability that each multi-level label is transferred to each multi-level label in the multi-level label set; Determine the total score based on the sum of the scores of all possible tag sequences; Determining a fourth sub-loss according to the score corresponding to the predicted label sequence and the total score; determining a second model loss according to the third sub-loss and the fourth sub-loss; The initial text annotation model is trained according to the second model loss to obtain the text annotation model.

17. The apparatus of claim 13, wherein: The first acquisition module is configured to: Obtaining the first sample text; Performing word segmentation processing on the first sample text to obtain a first word segmentation included in the first sample text, and determining the part of speech of the first word segmentation; Determining, from a multi-level tag set, a multi-level reference tag corresponding to the first participle according to the part of speech of the first participle and the parts of speech of adjacent participles of the first participle; According to the multi-level reference labels corresponding to the first word segmentation, characters in the first word segmentation are labeled to obtain the multi-level reference labels corresponding to the characters.

18. The apparatus of claim 17, wherein: The first acquisition module is configured to: Determine the first-level candidate label corresponding to the first word segmentation according to the mapping relationship between the part of speech and the first-level label; Determining a first reference tag from the first-level candidate tags based on the parts of speech of adjacent participles of the first participle; Matching the first word segmentation with labels of the remaining levels except the first-level label in a third candidate multi-level label to determine reference labels of the remaining levels, wherein the third candidate multi-level label refers to a multi-level label in the multi-level label set whose first-level label is the first reference label; The multi-level reference labels are determined according to the first reference label and the reference labels of the remaining levels.

19. The apparatus of claim 18, wherein: The first acquisition module is configured to: determining, based on the parts of speech of adjacent participles of the first participle, the sentence component of the first participle in the sentence to which it belongs; The first reference tag matching the sentence component is determined from the first-level candidate tags.

20. The apparatus of any one of claims 16 to 19, further comprising: A second acquisition module, used to acquire a second sample text; a third acquisition module, configured to perform a text-image demand analysis on the second sample text to obtain a multi-level label system, wherein each level of the multi-level label system includes at least one label; A fourth acquisition module is configured to acquire the multi-level tag set according to the multi-level tag system.

21. The apparatus of claim 20, wherein: The fourth acquisition module is configured to: Combining a primary tag in the multi-level tag system with a secondary tag under the primary tag to obtain a first tag; In the case where the second-level tag in the first tag has a third-level tag, the first tag is combined with the third-level tag to obtain a second tag, and the combination is continued until the combination is combined with the last-level tag to obtain an Nth tag, where N is an integer greater than 2; A multi-level tag set is acquired according to the first tag, the second tag, and up to the Nth tag.

22. The apparatus of claim 20, wherein: The third acquisition module is used to: Performing word segmentation processing on the second sample text to obtain second word segments contained in the second sample text; Performing a text-image demand analysis on the second participle to obtain demand information of the text-image scenario; The multi-level label system is acquired according to the demand information.

23. The apparatus of claim 22, wherein: The third acquisition module is used to: performing syntactic analysis and semantic recognition on the second participle respectively to determine the sentence component of the second participle in the sentence to which it belongs and the semantic information of the second participle; determining the demand information according to the sentence components and the semantic information; The multi-level label system is acquired according to the demand information.

24. A Wensheng diagram device, comprising: A first acquisition module is used to acquire the target text; a second acquisition module, configured to annotate the target text using a text annotation model to obtain a sequence annotation result corresponding to the target text, wherein the text annotation model is trained using the method according to any one of claims 1 to 11; The generation module is used to input the target text and the sequence annotation result into the text-graph model for processing to generate an image corresponding to the target text.

25. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 11, or to perform the method according to claim 12.

26. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 11, or to execute the method according to claim 12.

27. A computer program product comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the steps of the method according to any one of claims 1 to 11, or implements the steps of the method according to claim 12.

Images