Text generation method, device, equipment and medium

Abstract

The present application relates to the field of artificial intelligence technology, and specifically discloses a text generation method, apparatus, device, and medium. The text generation method obtains subject vocabulary and a pre-constructed knowledge graph; calculates the matching degree between the subject vocabulary and the graph nodes, and obtains at least one knowledge subgraph based on the graph nodes whose matching degree is greater than a matching degree threshold; obtains a keyword set and the path characteristics of each keyword in the keyword set for the knowledge subgraph based on the knowledge subgraph; inputs the keyword set and the path characteristics of each keyword in the keyword set for the knowledge subgraph into a pre-trained text generation model to obtain a target text. The method expands the vocabulary through the knowledge graph and the subject vocabulary to obtain a sufficient number of keywords, and then inputs the keywords into the text generation model to obtain a target text with richer content, thereby achieving the automatic generation of rich text information based on minimal vocabulary information.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and more specifically, to a text generation method, apparatus, device, and medium. Background Art

[0002] In recent years, with the continuous development of Internet technology, text generation has been widely used in various industries, including the automatic generation of news and product introductions. However, in text generation tasks, due to the limited input data information, the generated text lacks information content and the generation effect is poor.

[0003] Therefore, how to generate text with rich content is a key point that technicians in this field need to pay attention to. Summary of the Invention

[0004] To solve the above technical problems, embodiments of the present application provide a text generation method, apparatus, device, and medium to generate text with rich content.

[0005] In a first aspect, the present application provides a text generation method, comprising: obtaining a subject vocabulary and obtaining a pre-constructed knowledge graph; wherein the knowledge graph includes multiple graph nodes; calculating the matching degree between the subject vocabulary and the graph nodes to obtain at least one knowledge sub-graph based on the graph nodes whose matching degree is greater than a matching degree threshold; obtaining a keyword set and a path feature of each keyword in the keyword set for the knowledge sub-graph based on the knowledge sub-graph; inputting the keyword set and the path feature of each keyword in the keyword set for the knowledge sub-graph into a pre-trained text generation model to obtain a target text.

[0006] According to a preferred embodiment of the present invention, at least one knowledge subgraph is obtained based on graph nodes with a matching degree greater than a matching degree threshold, including: taking the graph nodes with a matching degree greater than the matching degree threshold as target nodes to obtain a target node set; obtaining graph nodes adjacent to the target node based on the knowledge graph, and adding the adjacent graph nodes to the target node set until the number of target nodes in the target node set reaches a preset number of nodes; and obtaining a knowledge subgraph based on the target nodes in the target node set and the relationship between the target nodes.

[0007] According to a preferred embodiment of the present invention, at least one knowledge subgraph is obtained based on graph nodes with a matching degree greater than a matching degree threshold, including: taking the graph node with a matching degree greater than the matching degree threshold as the starting node, and performing a random walk to other graph nodes in the knowledge graph according to the starting node; taking the graph nodes traversed in the random walk as target nodes until the number of target nodes reaches a preset number of nodes, thereby obtaining a target node set; and obtaining a knowledge subgraph based on the target nodes in the target node set and the relationship between the target nodes.

[0008] According to a preferred embodiment of the present invention, a text generation model is obtained by training an encoding and decoding model based on a self-attention mechanism, and a keyword set and the path features of each keyword in the keyword set for a knowledge subgraph are input into a pre-trained text generation model to obtain a target text, including: encoding the keywords in the keyword set to obtain word vectors corresponding to the keywords; fusing the word vectors corresponding to the keywords and the path features of the keywords for the knowledge subgraph to obtain feature vectors corresponding to each keyword; and processing the feature vectors through a self-attention mechanism to obtain the target text.

[0009] According to a preferred embodiment of the present invention, the feature vector is processed through a self-attention mechanism to obtain a target text, including: determining the attention weight corresponding to each keyword at the current time step based on the feature vector corresponding to each keyword and the output latent vector of the current time step; determining the text vocabulary corresponding to the current time step based on the attention weight corresponding to each keyword at the current time step, the output latent vector of the current time step and the input vector of the current time step; traversing all time steps, and obtaining the target text based on the text vocabulary output at each time step.

[0010] According to a preferred embodiment of the present invention, obtaining a pre-constructed knowledge graph includes: classifying subject vocabulary to obtain classification results corresponding to the subject vocabulary; and obtaining a knowledge graph that matches the classification results.

[0011] According to a preferred embodiment of the present invention, the method also includes: obtaining a word vector and a text vector for each character in the target text; wherein the word vector is used to represent the identification information of the character, and the text vector is used to represent the semantic information of the character in the target text; fusing the word vector and text vector of each character to obtain a fusion vector corresponding to each character; inputting the fusion vector corresponding to each character into a pre-trained text evaluation network, processing the fusion vector corresponding to each character through the text evaluation network, and obtaining text quality evaluation information corresponding to the target text.

[0012] In the second aspect, the present application provides a text generation device, including: an acquisition module for acquiring subject vocabulary and a pre-constructed knowledge graph; wherein the knowledge graph includes multiple graph nodes; a knowledge subgraph calculation module for calculating the matching degree between the subject vocabulary and the graph nodes, so as to obtain at least one knowledge subgraph based on the graph nodes whose matching degree is greater than the matching degree threshold; a keyword and feature acquisition module for obtaining a keyword set and the path features of each keyword in the keyword set for the knowledge subgraph based on the knowledge subgraph; a text generation module for inputting the keyword set and the path features of each keyword in the keyword set for the knowledge subgraph into a pre-trained text generation model to obtain the target text.

[0013] In a third aspect, the present application provides a computer device, which includes a memory and a processor; the memory is used to store a computer program; the processor is used to execute the computer program and implement the steps of the above-mentioned text generation method when executing the computer program.

[0014] In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor implements the steps of the above-mentioned text generation method.

[0015] The text generation method, apparatus, device and medium disclosed in the embodiment of the present application obtain subject vocabulary and a pre-constructed knowledge graph; wherein the knowledge graph includes a plurality of graph nodes; calculate the matching degree between the subject vocabulary and the graph nodes to obtain at least one knowledge subgraph based on the graph nodes whose matching degree is greater than a matching degree threshold; obtain a keyword set and the path characteristics of each keyword in the keyword set for the knowledge subgraph based on the knowledge subgraph; input the keyword set and the path characteristics of each keyword in the keyword set for the knowledge subgraph into a pre-trained text generation model to obtain a target text. The vocabulary is expanded through the knowledge graph and the subject vocabulary to obtain a sufficient number of keywords, and then the keywords are input into the text generation model to obtain a target text with richer content, thereby realizing the automatic generation of rich content text information based on very little vocabulary information. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings are incorporated into and constitute a part of the specification, illustrating embodiments consistent with the present application and, together with the specification, serving to explain the principles of the present application. It is obvious that the drawings described below are merely some embodiments of the present application, and a person of ordinary skill in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0017] Figure 1 Schematic diagram of the application environment of the text generation method provided in the embodiment of the present application;

[0018] Figure 2 is a flowchart of the text generation method provided in an embodiment of the present application;

[0019] Figure 3 is a schematic diagram of the knowledge graph provided in the embodiment of the present application;

[0020] Figure 4 is a schematic diagram of a knowledge graph provided by another embodiment of the present application;

[0021] Figure 5 This is a data flow diagram of the text generation method provided in an embodiment of the present application;

[0022] Figure 6 is a schematic block diagram of a text generation device provided in an embodiment of the present application;

[0023] Figure 7 It is a schematic block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] Exemplary embodiments will be described in detail herein, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numerals in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all embodiments consistent with the present application. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present application, as detailed in the appended claims.

[0025] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0026] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0027] It should also be noted that the term "plurality" used in this application refers to two or more. "And / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0028] The following describes some embodiments of the present application in detail with reference to the accompanying drawings. In the absence of conflict, the following embodiments and features therein may be combined with each other.

[0029] Figure 1 A schematic diagram of a system architecture of an operating environment of an exemplary embodiment of the present application is shown. Figure 1 As shown, the system may include a user terminal 110, a server 120, and a network. The user terminal 110 and the server 120 are connected to each other via a network communication. The network may include various connection types, such as wired or wireless communication links or optical fiber cables.

[0030] The user terminal 110 can be either hardware or software. If the user terminal 110 is hardware, it can be any electronic device, including but not limited to an in-vehicle smart terminal, a smartphone, a tablet computer, and a smart wristband. If the user terminal 110 is software, it can be installed in any of the electronic devices listed above.

[0031] Server 120 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0032] Among them, the text generation method provided in the embodiment of the present application can be used in a text generation device, and the text generation device can be integrated into a computer device, and the computer device can be a server 120 or a user terminal 110.

[0033] It should be understood that Figure 1 The number of user terminals 110 and servers 120 is merely illustrative and is only used to understand the embodiments of the present application. The specific number of user terminals 110 and servers 120 should be flexibly determined based on actual conditions.

[0034] See also Figure 2 , Figure 2 A flow chart of the text generation method provided in the embodiment of the present application. Figure 2 As shown, the method includes steps S210 to S240.

[0035] Step S210: Obtain subject vocabulary and obtain a pre-built knowledge graph; wherein the knowledge graph includes multiple graph nodes.

[0036] The subject vocabulary can be directly input by the user or obtained by identifying characteristic items. By obtaining the subject vocabulary, the text generation direction of the to-be-generated text is determined. The method for obtaining the subject vocabulary can be flexibly selected based on the actual application situation and is not limited in this embodiment. The subject vocabulary can be one or more.

[0037] For example, in the scenario of automatically generating a product introduction text, the subject vocabulary can be obtained by identifying the target product. For example, if the target product is identified as a beverage of brand A, the corresponding subject vocabulary includes "brand A" and "beverage".

[0038] For example, in the scenario of automatically generating a short video script, the theme vocabulary can be obtained by obtaining keywords input by the user regarding the theme, location, time of occurrence, main task, etc. of the script, and using the keywords as theme vocabulary. For example, the theme vocabulary obtained based on the user input can be "comedy", "weekend", "student", etc.

[0039] In some embodiments, obtaining a pre-constructed knowledge graph includes: classifying subject vocabulary to obtain classification results corresponding to the subject vocabulary; and obtaining a knowledge graph that matches the classification results.

[0040] It is understandable that different types of subject words correspond to different knowledge graphs. For example, if the acquired subject words include "introduction", "brand A", and "beverage", the classification results of these subject words can be product introductions, and then the knowledge graph of product introductions can be obtained.

[0041] It should be noted that the subject vocabulary can include multiple types of vocabulary, that is, it can match multiple knowledge graphs. For example, the subject vocabulary may include "Type A-vocabulary a1", "Type A-vocabulary a2", and "Type B-vocabulary b1". The classification results are "Type A-vocabulary a1" and "Type A-vocabulary a2" corresponding to Type A, and "Type B-vocabulary b1" corresponding to Type B. Therefore, the knowledge graph A matching Type A and the knowledge graph B matching Type B are obtained.

[0042] In some embodiments, before obtaining a pre-built knowledge graph, the process also includes constructing a knowledge graph.

[0043] Exemplarily, a vocabulary set is obtained based on historical text data; entity extraction and relationship extraction are performed on the vocabulary set to obtain a triple set; and a knowledge graph is obtained based on the triple set.

[0044] Among them, historical text data can include various types, such as product introductions, scenario stories, article reviews, etc. Therefore, the historical text data can be classified according to the type information of the historical text data, so as to obtain different types of vocabulary sets after performing preprocessing operations on different types of historical text data.

[0045] Optionally, the preprocessing operation performed on the historical text data includes stop word removal and word segmentation.

[0046] Because historical text data contains various noise information, such as modal particles, personal names, interjections, and stop words, as well as meaningless words, the historical text data is matched against a stop word library to filter out words that successfully match the stop word library, thereby achieving the effect of denoising the historical text data and obtaining standard text. The stop word library includes: modal particles, personal names, interjections, and stop words, as well as meaningless words.

[0047] The standard text is then pre-processed using word segmentation. Pre-defined word segmentation methods include, but are not limited to, using third-party word segmentation tools or algorithms. Common third-party word segmentation tools include, but are not limited to, the Stanford NLP word segmenter, the ICTClas word segmentation system, the ansj word segmentation tool, and the HanLP Chinese word segmentation tool. Word segmentation algorithms include, but are not limited to, the Maximum Forward Matching (MM) algorithm, the Reverse Direction Maximum Matching Method (RMM) algorithm, the Bi-directional Maximum Matching (BM) algorithm, the Hidden Markov Model (HMM), and the N-gram model.

[0048] Through stop word removal and word segmentation preprocessing, we obtain the vocabulary set corresponding to the historical text data. Then we perform entity extraction and relationship extraction on the vocabulary set.

[0049] For example, for a vocabulary set of product introduction type, the preset entity types include product name, product price, product appearance, product type and product sales volume, etc.; for a vocabulary set of situational story type, the preset entity types include place name, scene atmosphere description, character emotion description, character appearance description and action description, etc.

[0050] Furthermore, relationships between entities are obtained from the sentence information in the historical text data for relationship extraction. Relationships between entities refer to associations between different entities, and associations between entities include one-to-one, one-to-many, and many-to-many. For example, after obtaining entity information from the historical text data, a set of triples is obtained by obtaining a preset triple rule and then adding the relationships between the entities to the entity information according to the preset triple rule.

[0051] For example, for historical text data of a product introduction type, the historical text data includes a sentence "Brand A's beverage is a juice-type beverage, which quenches thirst and replenishes energy."

[0052] The entity information contained in this statement is as follows:

[0053] Product brand entity (BRA): Brand A;

[0054] Product first type entity (TYP): beverage;

[0055] Second type of commodity entity (subT): juice type;

[0056] Functional entity of product (FUNC): quench thirst and replenish energy.

[0057] Furthermore, based on the preset triple rules, the relationships between entities added to the entity information are as follows: "Quenching thirst, replenishing energy" is_FUNC_of "product first category entity", "product second category entity" is_subT_of "product first category entity", and "product brand entity" is_BRA_of "product first category entity". Among them, is_FUNC_of, is_subT_of, and is_BRA_of are preset triple rules, which, together with the product brand entity, product first category entity, product second category entity, and product function entity, form multiple triples of information.

[0058] Through all the triple information of historical text data, a triple set is obtained, and the corresponding knowledge graph is obtained based on the triple set.

[0059] Exemplarily, the knowledge graph to be generated includes nodes and edges. A node is corresponding entity information, represented by a globally unique identifier. For example, product name, product price, product appearance, product category, and product sales volume can all serve as nodes in the knowledge graph. An edge is used to connect two nodes and refers to a directed line connecting nodes in the knowledge graph, used to represent the relationship between different nodes. For example, the entity relationship between product name, product price, product appearance, product category, and product sales volume can all serve as an edge in the knowledge graph.

[0060] Step S220: Calculate the matching degree between the subject vocabulary and the graph nodes to obtain at least one knowledge subgraph based on the graph nodes whose matching degree is greater than the matching degree threshold.

[0061] Based on the pre-built knowledge graph, other words that match the subject vocabulary are obtained, such as synonyms, hypernyms, hyponyms, etc. of the subject vocabulary, so as to expand the initial vocabulary required for text generation, enrich the vocabulary information, and facilitate subsequent text generation.

[0062] Optionally, when the subject vocabulary is multiple words, they can correspond to different knowledge graphs respectively, and each subject vocabulary can obtain multiple knowledge subgraphs according to the corresponding knowledge graph.

[0063] After obtaining the subject vocabulary, the subject vocabulary can be matched according to the knowledge graph corresponding to the subject vocabulary to obtain the corresponding knowledge subgraph. The vocabulary corresponding to each subgraph node in the knowledge subgraph is a vocabulary whose matching degree with the subject vocabulary is greater than the matching degree threshold.

[0064] Exemplarily, the matching degree between the topic vocabulary and the entities corresponding to each graph node in the knowledge graph is calculated. When the matching degree is larger, it indicates that the corresponding entity is more similar to the topic vocabulary; when the matching degree is smaller, it indicates that the corresponding entity has a greater difference from the topic vocabulary. Therefore, in order to improve the relevance between the generated text and the topic vocabulary, the graph nodes with a matching degree greater than the matching degree threshold are selected as target nodes, and a knowledge subgraph is obtained based on the target nodes.

[0065] For example, when the topic vocabulary is "fruit", according to the knowledge graph, its corresponding synonym can be "raw fruit", its corresponding hypernym can be "food", and its corresponding hyponym can be "apple". Then, through the knowledge graph, the obtained target nodes include "raw fruit, food, apple". Then, the corresponding knowledge subgraph is obtained according to the target nodes.

[0066] In some embodiments, obtaining at least one knowledge subgraph according to the graph nodes with a matching degree greater than the matching degree threshold includes: taking the graph nodes with a matching degree greater than the matching degree threshold as target nodes to obtain a target node set; obtaining the graph nodes adjacent to the target nodes according to the knowledge graph, and adding the adjacent graph nodes to the target node set until the number of target nodes in the target node set reaches the preset node number; obtaining a knowledge subgraph according to the target nodes in the target node set and the relationships between the target nodes.

[0067] The graph nodes adjacent to the target nodes refer to the nodes having a connection relationship with the target nodes.

[0068] Among them, the preset node number can be set in advance by relevant personnel. For example, the user can directly limit the number of target nodes; it can also be matched according to the length of the text to be generated. For example, when the number of characters set for the text to be generated is 100, the corresponding preset node number can be 10, and when the number of characters set for the text to be generated is 200, the corresponding preset node number can be 20.

[0069] It can be understood that the preset node number can be a specific value or a numerical range.

[0070] Optionally, obtain the graph nodes adjacent to the target nodes, and take the adjacent graph nodes as target nodes to expand the target nodes once. Further, detect whether the number of the expanded target nodes reaches the preset node number. If it does not reach the preset node number, obtain the graph nodes adjacent to the expanded target nodes, and take the adjacent graph nodes as target nodes to expand the target nodes again. If the preset node number is reached, obtain all the target nodes and the relationships between the target nodes to obtain a knowledge subgraph.

[0071] Please refer to Figure 3 For example, the target nodes include node A and node B connected to node A, node A is connected to node C, node B is connected to node D, and node D is connected to node E. When the preset number of nodes is 4, the graph nodes adjacent to node A include node B and node C, and the graph nodes adjacent to node B include node A and node D. Through deduplication processing, the target nodes obtained include node A, node B, node C, and node D, and the number of expanded target nodes is 4, reaching the preset number of nodes. Then, the knowledge subgraph corresponding to node A, node B, node C, and node D is obtained through the target nodes and the relationship between the target nodes.

[0072] In some embodiments, at least one knowledge subgraph is obtained based on graph nodes with a matching degree greater than a matching degree threshold, including: taking the graph node with a matching degree greater than the matching degree threshold as the starting node, and performing a random walk to other graph nodes in the knowledge graph based on the starting node; taking the graph nodes traversed in the random walk as target nodes until the number of target nodes reaches a preset number of nodes, thereby obtaining a target node set; and obtaining a knowledge subgraph based on the target nodes in the target node set and the relationship between the target nodes.

[0073] The starting node refers to the first node traversed during a random walk.

[0074] When a random walk is performed starting from the starting node, other nodes can be randomly traversed in multiple different directions. For example, the starting node is node A, and node A is connected to nodes B and C. Therefore, when a random walk is performed starting from node A, other nodes can be randomly traversed in the direction of node B, and other nodes can also be randomly traversed in the direction of node C. It can be understood that during the random walk, all paths that have not been traversed under the current node can be traversed, that is, there are multiple walk paths in a single walk.

[0075] See also Figure 4 For example, the target node includes node A, and other graph nodes in the knowledge subgraph corresponding to node A are as follows: Figure 4 As shown in the figure. Taking node A as the starting node, the first random walk is performed towards other nodes. The walk direction during the walk is towards node B. The walk paths in the direction of node B include A→B→E and A→B→F. Therefore, the traversed nodes are nodes A, B, E, and F. Taking node A as the starting node, the second random walk is performed towards other nodes. The walk direction during the walk is towards node C. The walk paths in the direction of node C include A→C→G. Therefore, the traversed nodes are nodes A, C, and G.

[0076] Starting from the target node, a random walk is performed toward other nodes, adding the graph nodes traversed during the walk as target nodes until the number of target nodes reaches the preset number of nodes. It is understandable that through the random walk approach, different nodes can be traversed during different walks, even if the starting node is the same. Furthermore, when the subject vocabulary input in each text generation task is the same, different knowledge subgraphs can be obtained, ensuring the diversity of the generated text.

[0077] Step S230: According to the knowledge subgraph, obtain a keyword set and a path feature of each keyword in the keyword set with respect to the knowledge subgraph.

[0078] For example, the entities corresponding to all graph nodes in the knowledge subgraph can be used as keywords to obtain a keyword set. It is also possible to filter the entities corresponding to all graph nodes in the knowledge subgraph, and use the entities corresponding to the filtered graph nodes as keywords. For example, the matching degree between the subject vocabulary and the entities of all graph nodes in the corresponding knowledge subgraph is calculated, and entities with a matching degree greater than a matching degree threshold are used as keywords. By filtering the graph nodes, the quality of the output text is guaranteed.

[0079] Exemplarily, the path feature of the keyword is obtained based on the position information of the graph node of the entity corresponding to each keyword in the keyword set in the knowledge subgraph and the connection relationship between the graph nodes.

[0080] For example, entity A corresponds to node A, node A is connected to node B and node C respectively, and the connection relationship between node A and node B is relationship 1, and the connection relationship between node A and node C is relationship 2. Then, according to the preset calculation rules, node B, node C, relationship 1 and relationship 2 are calculated to obtain the path characteristics corresponding to entity A.

[0081] A knowledge subgraph is obtained through the subject vocabulary, and multiple keywords are obtained based on the knowledge subgraph, which can enrich the vocabulary of the input text generation model.

[0082] Step S240: Input the keyword set and the path feature of each keyword in the keyword set with respect to the knowledge subgraph into a pre-trained text generation model to obtain a target text.

[0083] In some embodiments, a text generation model may be pre-trained, and the text generation model is used to generate target text based on an input keyword set and a path feature corresponding to each keyword.

[0084] It is understandable that since the text generation method provided in this embodiment generates text based on very few subject words, it results in a lack of sample data during the training process of the initial text generation model. Therefore, in some embodiments, a pre-trained language model can be used to train the initial text generation model. For example, the text generation model is trained based on the training weights of the pre-trained language model so that the text generation model can reach training convergence faster. Pre-trained language models include but are not limited to T5 (Transfer Text-to-Text Transformer) models, GPT3 (Generative Pre-trained Transformer 3) models, etc.

[0085] In some embodiments, the text generation model is obtained by training an encoding and decoding model based on a self-attention mechanism, and a keyword set and the path features of each keyword in the keyword set for the knowledge subgraph are input into a pre-trained text generation model to obtain a target text, including: encoding the keywords in the keyword set to obtain word vectors corresponding to the keywords; fusing the word vectors corresponding to the keywords and the path features of the keywords for the knowledge subgraph to obtain feature vectors corresponding to each keyword; processing the feature vectors through the self-attention mechanism to obtain the target text.

[0086] Optionally, the text generation model is a Seq2Seq model, the input of which is a sequence (a set of keywords and path features of the keywords), and the output is also a sequence (generated text). For example, the text generation model uses RNN, LSTM, BLSTM, or GRU, etc. The embodiments of this application do not specifically limit the form of the text generation model.

[0087] The Seq2Seq model includes an encoding layer, which maps discrete input words into continuous vectors. In other words, it maps words into an embedding space to obtain word vectors. This compresses sparse words into dense word vectors, making it easier for computers to process text. The decoding layer, based on the output of the encoder and combined with a self-attention mechanism, outputs coherent text information.

[0088] The self-attention mechanism is a variant of the attention mechanism that reduces reliance on external information and is better at capturing the internal correlations of data or features. It addresses long-distance dependencies by calculating the mutual influence between words. The attention mechanism, based on a weighted attention coefficient, can be applied to the decoding phase of a Seq2Seq model to weight the latent vectors generated during decoding. This weights the latent vectors, making them more focused on the core semantic information of the text. This results in the Seq2Seq model tending to output information indicated by latent vectors with higher attention coefficients.

[0089] In some embodiments, the feature vector is processed through a self-attention mechanism to obtain a target text, including: determining the attention weight corresponding to each keyword at the current time step based on the feature vector corresponding to each keyword and the output latent vector of the current time step; determining the text vocabulary corresponding to the current time step based on the attention weight corresponding to each keyword at the current time step, the output latent vector of the current time step, and the input vector of the current time step; traversing all time steps, and obtaining the target text based on the text vocabulary output at each time step.

[0090] The decoding layer of the text generation model decodes the feature vector corresponding to each keyword to produce an output latent vector, which is used to determine the text vocabulary for the corresponding time step. The text vocabulary refers to the words in the target text. During the target text generation process, the target text vocabulary is generated on a time-step basis to ensure dependencies between words in the target text.

[0091] Correspondingly, the decoding layer also outputs the output latent vector for each time step according to the time step, ensuring that the output latent vector of the current time step is used to determine the text word corresponding to the current time step. That is, the decoding layer outputs the latent vector corresponding to each text word in the target text one by one, and the process of determining a text word is called a time step. Since the decoding layer determines the output latent vector corresponding to the text word according to the arrangement order of the text words in the target text, the output latent vector output by the decoding layer at time step t (or the t-th time step, where t ≥ 1 and t is a positive integer) corresponds to the t-th word in the target text.

[0092] In some embodiments, the decoding layer may be a long short-term memory network. Specifically, the decoding layer may be one or more layers of a long short-term memory network, which is not specifically limited here. In some embodiments, the decoding layer may also be a decoder in a transformer network (Transformer model). Of course, in other embodiments, the decoding layer may also be other neural networks for text generation, which is not specifically limited here.

[0093] In some embodiments, after obtaining the target text, the text generation method provided by the embodiments of the present application further includes: obtaining the word vector and text vector of each character in the target text; wherein, the word vector is used to represent the identification information of the character, and the text vector is used to represent the semantic information of the character in the target text; fusing the word vector and text vector of each character to obtain a fused vector corresponding to each character; inputting the fused vector corresponding to each character into a pre-trained text evaluation network, and processing the fused vector corresponding to each character through the text evaluation network to obtain text quality evaluation information corresponding to the target text.

[0094] Optionally, in this embodiment, the text quality evaluation information corresponding to the target text can be obtained through a text evaluation model. In this embodiment, the text evaluation model includes a feature extraction network and a text evaluation network. Obtain the fused vector corresponding to each character of the target text through the feature extraction network; obtain the text quality evaluation information corresponding to the target text through the text evaluation network.

[0095] Among them, the word vector is used to represent the identification information of the character, and different characters correspond to different word vectors; for example, the word vector corresponding to "文" is different from the word vector corresponding to "本", and different characters are uniquely identified by different word vectors; the text vector is used to represent the semantic information of the character in the target text, and the closer the semantics of the characters are, the closer the corresponding text vectors are.

[0096] Exemplarily, in the process of fusing the word vector and text vector corresponding to each character, first perform a summation process on the word vector and text vector corresponding to each character to obtain a sum vector corresponding to each character, and then perform a fusion process according to the sum vector corresponding to each character to obtain a fused vector, where the fused vector represents the semantic information corresponding to each character.

[0097] Further, input the fused vector corresponding to each character into the averaging layer of the trained text evaluation network. The averaging layer of the trained text evaluation network performs an averaging process on the fused vector corresponding to each character to obtain a text quality evaluation vector corresponding to the target text; and input the obtained text quality evaluation vector into the evaluation information acquisition layer of the trained text evaluation network. The evaluation information acquisition layer of the trained text evaluation network performs a normalization process on the text quality evaluation vector to obtain text quality evaluation information corresponding to the target text.

[0098] Exemplarily, the text evaluation network can calculate the smoothness of the target text through the text quality evaluation vector, score the target text according to the smoothness, and obtain the text quality evaluation information. The smoothness can be represented by a probability, and its range is 0 to 1.

[0099] Among them, the smoothness refers to that each word in the target text can be connected smoothly.

[0100] By evaluating the target text, we can obtain the text quality evaluation information of the target text, and then we can understand the text generation situation based on the text quality evaluation information. When the text quality evaluation information indicates that the quality of the target text is poor, we can timely optimize the knowledge graph, text generation model, etc. in the text generation process.

[0101] See also Figure 5 , Figure 5 A data flow diagram of a text generation method provided in another embodiment of the present application.

[0102] like Figure 5 As shown, a subject vocabulary is obtained, and a knowledge graph of a corresponding type is obtained according to the type of the subject vocabulary. The graph nodes in the knowledge graph are matched with the subject vocabulary to obtain a knowledge subgraph based on the successfully matched graph nodes. The entity vocabulary corresponding to the graph nodes in the knowledge subgraph is then used as a keyword to obtain a keyword set, and the path feature of each keyword with respect to the knowledge subgraph is obtained based on the position information of each keyword in the knowledge subgraph. Furthermore, the keyword set and the path feature of each keyword with respect to the knowledge subgraph are input into a text generation model so that the text generation model outputs the target text.

[0103] The text generation method provided by the present application obtains a subject vocabulary and a pre-constructed knowledge graph; wherein the knowledge graph includes multiple graph nodes; calculates the matching degree between the subject vocabulary and the graph nodes to obtain at least one knowledge subgraph based on the graph nodes whose matching degree is greater than a matching degree threshold; obtains a keyword set and the path characteristics of each keyword in the keyword set for the knowledge subgraph based on the knowledge subgraph; inputs the keyword set and the path characteristics of each keyword in the keyword set for the knowledge subgraph into a pre-trained text generation model to obtain a target text. The method expands the vocabulary through the knowledge graph and the subject vocabulary to obtain a sufficient number of keywords, and then inputs the keywords into the text generation model to obtain a target text with richer content, thereby achieving the automatic generation of rich text information based on minimal vocabulary information.

[0104] See also Figure 6 , Figure 6 This is a schematic block diagram of a text generation device provided in one embodiment of the present application. The text generation device can be configured in a server or computer device to execute the aforementioned text generation method.

[0105] like Figure 6 As shown, the text generation device 600 includes: an acquisition module 610, a knowledge subgraph calculation module 620, a keyword and feature acquisition module 630 and a text generation module 640.

[0106] The acquisition module 610 is used to acquire a subject vocabulary and obtain a pre-built knowledge graph; wherein the knowledge graph includes a plurality of graph nodes;

[0107] The knowledge subgraph calculation module 620 is used to calculate the matching degree between the subject words and the graph nodes, so as to obtain at least one knowledge subgraph according to the graph nodes whose matching degree is greater than the matching degree threshold;

[0108] The keyword and feature acquisition module 630 is used to obtain a keyword set and a path feature of each keyword in the keyword set for the knowledge subgraph based on the knowledge subgraph;

[0109] The text generation module 640 is used to input the keyword set and the path feature of each keyword in the keyword set with respect to the knowledge subgraph into a pre-trained text generation model to obtain a target text.

[0110] In some embodiments, based on the aforementioned scheme, at least one knowledge subgraph is obtained according to the graph nodes whose matching degree is greater than the matching degree threshold, including: taking the graph nodes whose matching degree is greater than the matching degree threshold as the target nodes to obtain a target node set; obtaining the graph nodes adjacent to the target nodes according to the knowledge graph, and adding the adjacent graph nodes to the target node set until the number of target nodes in the target node set reaches a preset number of nodes; obtaining a knowledge subgraph according to the target nodes in the target node set and the relationship between the target nodes.

[0111] In some embodiments, based on the aforementioned scheme, at least one knowledge subgraph is obtained according to the graph nodes whose matching degree is greater than the matching degree threshold, including: taking the graph node whose matching degree is greater than the matching degree threshold as the starting node, and performing a random walk to other graph nodes in the knowledge graph according to the starting node; taking the graph nodes traversed in the random walk as target nodes until the number of target nodes reaches a preset number of nodes, and obtaining a target node set; obtaining a knowledge subgraph according to the target nodes in the target node set and the relationship between the target nodes.

[0112] In some embodiments, based on the aforementioned scheme, the text generation model is obtained by training an encoding and decoding model based on a self-attention mechanism, and the keyword set and the path features of each keyword in the keyword set for the knowledge subgraph are input into the pre-trained text generation model to obtain the target text, including: encoding the keywords in the keyword set to obtain the word vectors corresponding to the keywords; fusing the word vectors corresponding to the keywords and the path features of the keywords for the knowledge subgraph to obtain the feature vectors corresponding to each keyword; processing the feature vectors through the self-attention mechanism to obtain the target text.

[0113] In some embodiments, based on the aforementioned scheme, the feature vector is processed through a self-attention mechanism to obtain a target text, including: determining the attention weight corresponding to each keyword at the current time step based on the feature vector corresponding to each keyword and the output latent vector of the current time step; determining the text vocabulary corresponding to the current time step based on the attention weight corresponding to each keyword at the current time step, the output latent vector of the current time step, and the input vector of the current time step; traversing all time steps, and obtaining the target text based on the text vocabulary output at each time step.

[0114] In some embodiments, based on the aforementioned scheme, a pre-constructed knowledge graph is obtained, including: classifying the subject vocabulary to obtain classification results corresponding to the subject vocabulary; and obtaining a knowledge graph that matches the classification results.

[0115] In some embodiments, based on the aforementioned scheme, a word vector and a text vector are obtained for each character in the target text; wherein the word vector is used to represent the identification information of the character, and the text vector is used to represent the semantic information of the character in the target text; the word vector and the text vector of each character are fused to obtain a fused vector corresponding to each character; the fused vector corresponding to each character is input into a pre-trained text evaluation network, and the fused vector corresponding to each character is processed by the text evaluation network to obtain text quality evaluation information corresponding to the target text.

[0116] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0117] The methods and apparatus of the present application can be used in a wide variety of general-purpose or specialized computing system environments or configurations, such as personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above.

[0118] For example, the above method and apparatus may be implemented in the form of a computer program. The computer program may be implemented in the form of a computer program. Figure 7 Runs on the computer equipment shown.

[0119] See also Figure 7 , Figure 7 1 is a schematic diagram of a computer device provided in an embodiment of the present application. The computer device may be a server or a terminal.

[0120] like Figure 7As shown, the computer device 700 includes a processor 710 , a memory 730 , and a network interface 740 connected via a system bus 720 , wherein the memory 730 may include a non-volatile storage medium and an internal memory.

[0121] The non-volatile storage medium may store an operating system 750 and a computer program 760. The computer program 760 includes program instructions, which, when executed, may enable the processor 710 to perform any text generation method.

[0122] The processor 710 is used to provide computing and control capabilities and support the operation of the entire computer device 700.

[0123] The internal memory 730 provides an environment for the operation of the computer program 760 in the non-volatile storage medium. When the computer program 760 is executed by the processor 710, the processor 710 can execute any text generation method.

[0124] The network interface 740 is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that the structure of the computer device 700 is merely a block diagram of a portion of the structure related to the solution of the present application, and does not constitute a limitation on the computer device 700 to which the solution of the present application is applied. Specifically, the computer device 700 may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0125] It should be understood that the processor 710 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor 710 may be a microprocessor or any conventional processor.

[0126] The processor 710 is configured to execute a computer program 760 stored in the memory to implement any text generation method provided in the embodiments of the present application.

[0127] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed, any one of the text generation methods provided in the embodiments of the present application is implemented.

[0128] The computer-readable storage medium may be an internal storage unit of the computer device described in the aforementioned embodiment, such as a hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) card, a flash memory card, etc., provided on the computer device.

[0129] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present application, and such modifications or substitutions should be included in the scope of protection of the present application. Therefore, the scope of protection of the present application should be based on the scope of protection of the claims.

Claims

1. A text generation method, characterized in that: The method comprises: Obtaining a subject vocabulary and a pre-built knowledge graph; wherein the knowledge graph includes a plurality of graph nodes; Calculating the matching degree between the subject vocabulary and the graph nodes, so as to obtain at least one knowledge subgraph according to the graph nodes whose matching degree is greater than a matching degree threshold; According to the knowledge subgraph, a keyword set and a path feature of each keyword in the keyword set with respect to the knowledge subgraph are obtained; The keyword set and the path features of each keyword in the keyword set for the knowledge subgraph are input into a pre-trained text generation model to obtain a target text; wherein, The text generation model is a Seq2Seq model, and the encoding layer of the text generation model maps the input keywords into continuous vectors, so as to map the keywords into an embedding space to obtain word vectors of the keywords; The word vector corresponding to the keyword and the path feature of the keyword for the knowledge subgraph are fused to obtain a feature vector corresponding to each keyword; The decoding layer of the text generation model performs decoding processing on the feature vector corresponding to each keyword to obtain an output latent vector. The output latent vector is used to determine the text vocabulary of the corresponding time step, and the decoding layer outputs the output latent vector according to the time step. The attention mechanism is applied to the decoding layer of the text generation model based on a weighting mechanism of the attention coefficient to weight the output latent vector generated in the decoding part, so that the output latent vector is more focused on the core semantic information of the text. Traverse all time steps and get the target text based on the text vocabulary output at each time step.

2. The method according to claim 1, characterized in that The step of obtaining at least one knowledge subgraph based on graph nodes having a matching degree greater than a matching degree threshold comprises: Taking the graph nodes whose matching degree is greater than the matching degree threshold as target nodes, and obtaining a target node set; Obtaining graph nodes adjacent to the target node according to the knowledge graph, and adding the adjacent graph nodes to the target node set until the number of target nodes in the target node set reaches a preset number of nodes; The knowledge subgraph is obtained according to the target nodes in the target node set and the relationships between the target nodes.

3. The method according to claim 1, characterized in that The step of obtaining at least one knowledge subgraph based on graph nodes having a matching degree greater than a matching degree threshold comprises: Taking the graph node with a matching degree greater than a matching degree threshold as the starting node, and performing a random walk to other graph nodes in the knowledge graph based on the starting node; The graph nodes traversed in the random walk are used as target nodes until the number of target nodes reaches a preset number of nodes, thereby obtaining a target node set; The knowledge subgraph is obtained according to the target nodes in the target node set and the relationships between the target nodes.

4. The method according to claim 1, wherein The text generation model is obtained by training an encoding / decoding model based on a self-attention mechanism, and the keyword set and the path feature of each keyword in the keyword set for the knowledge subgraph are input into the pre-trained text generation model to obtain the target text, further comprising: Encoding the keywords in the keyword set to obtain word vectors corresponding to the keywords; The word vector corresponding to the keyword and the path feature of the keyword for the knowledge subgraph are fused to obtain a feature vector corresponding to each keyword; The feature vector is processed through a self-attention mechanism to obtain the target text.

5. The method according to claim 4, characterized in that The process of processing the feature vector by the self-attention mechanism to obtain the target text includes: Determine the attention weight corresponding to each keyword at the current time step according to the feature vector corresponding to each keyword and the output latent vector of the current time step; Determine the text vocabulary corresponding to the current time step according to the attention weight corresponding to each keyword at the current time step, the output latent vector of the current time step, and the input vector of the current time step; Traverse all time steps and get the target text based on the text vocabulary output at each time step.

6. The method according to any one of claims 1 to 5, characterized in that The obtaining of a pre-built knowledge graph includes: Classifying the subject vocabulary to obtain classification results corresponding to the subject vocabulary; Obtain a knowledge graph that matches the classification result.

7. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Obtaining a word vector and a text vector for each character in the target text; wherein the word vector is used to represent identification information of the character, and the text vector is used to represent semantic information of the character in the target text; Fusing the word vector and the text vector of each character to obtain a fused vector corresponding to each character; The fusion vector corresponding to each character is input into a pre-trained text evaluation network, and the fusion vector corresponding to each character is processed by the text evaluation network to obtain text quality evaluation information corresponding to the target text.

8. A text generation device, characterized in that: The device comprises: An acquisition module, configured to acquire a subject vocabulary and a pre-built knowledge graph; wherein the knowledge graph includes a plurality of graph nodes; A knowledge subgraph calculation module, configured to calculate the degree of matching between the subject vocabulary and the graph nodes, so as to obtain at least one knowledge subgraph based on the graph nodes having a degree of matching greater than a matching threshold; A keyword and feature acquisition module, configured to obtain a keyword set and a path feature of each keyword in the keyword set for the knowledge subgraph based on the knowledge subgraph; A text generation module, configured to input the keyword set and the path feature of each keyword in the keyword set with respect to the knowledge subgraph into a pre-trained text generation model to obtain a target text; in, The text generation model is a Seq2Seq model, and the encoding layer of the text generation model maps the input keywords into continuous vectors, so as to map the keywords into an embedding space to obtain word vectors of the keywords; The word vector corresponding to the keyword and the path feature of the keyword for the knowledge subgraph are fused to obtain a feature vector corresponding to each keyword; The decoding layer of the text generation model performs decoding processing on the feature vector corresponding to each keyword to obtain an output latent vector. The output latent vector is used to determine the text vocabulary of the corresponding time step, and the decoding layer outputs the output latent vector according to the time step. The attention mechanism is applied to the decoding layer of the text generation model based on a weighting mechanism of the attention coefficient to weight the output latent vector generated in the decoding part, so that the output latent vector is more focused on the core semantic information of the text. Traverse all time steps and get the target text based on the text vocabulary output at each time step.

9. A computer device, characterized in that: The computer device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and implement the text generation method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to implement the text generation method according to any one of claims 1 to 7.

Images