Method for generating relational data by means of knowledge graph

Abstract

The invention relates to a method for generating relational data by means of a knowledge graph, and belongs to the technical field of artificial intelligence. According to the method, semantic association and importance among nodes can be effectively captured by using a Graph Transform and utilizing a multi-head attention mechanism and neighborhood aggregation of a feedforward neural network, so that the characteristics of an original structure are reserved in a newly generated map, and the structure of the newly generated map and the structure of the original map are kept consistent; data enhancement is carried out by using a large language model (LLM) in combination with a prompt project, an enhancement strategy is applied in context and entity levels, and it is ensured that the content of an enhanced text is consistent with reality logic through a replacement strategy while an original triple structure is kept. And the pre-training model is finely adjusted to adapt to entity recognition and relation extraction tasks, so that the reliability and the practical application value of the generated content are further improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and specifically relates to a method for generating relational data with the help of a knowledge graph. Background Art

[0002] In the current field of knowledge graph generation, there are various graph generation technologies, including rule-based generation, graph embedding methods, and deep learning models. However, due to the limitations of their respective principles, they face a series of problems in the process of generating graphs. Rule-based generation methods rely on predefined rules and lack flexibility, which makes the generated graphs easy to simplify or deform in structure, and it is difficult to maintain the complex relationships of the original graphs. Graph embedding methods generate new graphs by mapping graph structures to low-dimensional spaces, but in this process, the detailed relationships of the graphs are often ignored. The generated graph structure is often inconsistent with the original graph, and it is easy to introduce fictitious nodes or relationships. Generative models based on deep learning (such as GAN, variational autoencoders, etc.) rely on data distribution to generate nodes and relationships, but due to the randomness of the content generated by the model, it is difficult to ensure that the new graph is consistent with the original graph structure, and may generate fictitious relationships or entities that do not conform to reality. These problems make it difficult for the generated knowledge graphs to ensure consistency and reliability in practical applications. These problems limit the availability of new knowledge graphs. Summary of the invention

[0003] 1. Technical issues to be resolved

[0004] The technical problems to be solved by the present invention are: 1. To solve the problem that the new map generated in the prior art has significant structural deviations from the original map and is difficult to maintain consistency; 2. To solve the problem that the content of the map generated in the prior art is often inconsistent with the real scene and lacks authenticity and usability.

[0005] (II) Technical solution

[0006] In order to solve the above technical problems, the present invention provides a method for generating relational data by means of a knowledge graph, comprising the following steps:

[0007] Step 1: Use Graph Transformer to generate natural language text from the knowledge graph. Graph Transformer is a graph neural network model based on the graph attention mechanism. It uses the attention mechanism to transfer and aggregate information in the graph structure and convert this information into natural language description.

[0008] Step 2: Perform data enhancement driven by a large language model on the natural language text generated in step 1 to generate enhanced text;

[0009] Among them, the natural language text data generated from the knowledge graph is enhanced through a dual enhancement strategy at the context level and the entity level to ensure that the generated text is consistent with reality in terms of structure and content. First, preliminary enhancement at the context level is adopted, that is, by adjusting the sentence length, vocabulary usage, clause structure and text style to generate different text expressions, while maintaining the triple structure and increasing the diversity of the text; then enhancement is performed at the entity level, using LLM to replace the entities in the original sentences of the natural language text with other entities of the same type, enriching the diversity of the data and ensuring that the content conforms to the real logic;

[0010] Step 3: Extract triples from the enhanced text and construct a new knowledge graph;

[0011] Among them, the triples of enhanced text are generated through fine-tuning technology, and a new knowledge graph is generated based on the generated triples; among them, by fine-tuning the BERT model on the OntoNotes 5.0 and TACRED datasets, the BERT model is made adaptable to entity recognition and relationship extraction tasks. The fine-tuned BERT model can recognize entities and their categories in the text, and divide them into multiple triples according to categories; and then a new knowledge graph is generated based on each triple. The new knowledge graph is consistent with the original knowledge graph in structure, but achieves richness and diversity in content.

[0012] (III) Beneficial effects

[0013] The present invention proposes a method for generating relational data by means of a knowledge graph. The implementation of this method can achieve the following effects:

[0014] 1. Ensure that the generated graph structure is similar to the original graph: By using Graph Transformer, taking advantage of the multi-head attention mechanism and neighborhood aggregation of the feedforward neural network, the semantic association and importance between nodes can be effectively captured, thereby retaining the characteristics of the original structure in the newly generated graph, ensuring that the newly generated graph structure is consistent with the original graph structure.

[0015] 2. Ensure that the generated graph content is consistent with real-world scenarios: By using a large language model (LLM) combined with prompt engineering for data enhancement, applying enhancement strategies at the context and entity levels, and maintaining the original triple structure, the replacement strategy ensures that the enhanced text content is consistent with real-world logic. Fine-tuning the pre-trained model to adapt to entity recognition and relationship extraction tasks further improves the reliability and real-world application value of the generated content. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is the overall framework diagram of the method of the present invention;

[0017] Figure 2A flow chart of natural text sentences in the knowledge graph generated by the method of the present invention;

[0018] Figure 3 A flow chart of data enhancement using a large language model in the method of the present invention;

[0019] Figure 4 A flowchart for extracting triples and constructing a knowledge graph in the method of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below in conjunction with the drawings and examples.

[0021] The present invention proposes a method for generating relational data with the help of a knowledge graph, that is, generating a new graph according to the structure of the existing knowledge graph, ensuring that the generated graph structure is similar to the original graph, and the content in the graph is in line with reality and does not contain fictitious and unreasonable information. Through the present invention, it is expected to break through the limitations of current knowledge graph generation technology, provide more reliable and practical data support for the application field of knowledge graphs, promote the widespread application of knowledge graph technology in various industries, and thus promote the development of intelligent decision-making and services.

[0022] The overall framework of this method is as follows Figure 1 shown.

[0023] Step 1. Use Graph Transformer to generate natural language text from the knowledge graph. GraphTransformer is a graph neural network model based on the graph attention mechanism, which can extract accurate natural sentences from the knowledge graph. Unlike traditional graph neural networks, Graph Transformer uses an attention mechanism instead of a simple summation operation in the process of neighborhood aggregation, which enables the model to capture the relationship and importance between nodes more flexibly. The Graph Transformer model also includes components such as normalization and MLP feedforward neural networks to enhance its expressiveness and performance. The main advantage of GraphTransformer is that it can use the attention mechanism to effectively transfer and aggregate information in the graph structure, thereby better capturing the complex relationships and importance between entities and converting this information into natural language descriptions.

[0024] The process of using Graph Transformer to generate natural language text from a knowledge graph is as follows: The triples in the original knowledge graph (consisting of a subject, a relationship, and an object) are abstracted into representations of nodes and edges, and input into GraphTransformer for processing. Graph Transformer uses a multi-head attention mechanism to identify the semantic relationship and importance between each node and its neighboring nodes, and updates the representation of the node. Nonlinear transformations are then performed through normalization and feedforward neural networks to enhance the expressiveness and stability of the GraphTransformer model. The updated node representation is input into the decoder, which gradually predicts the next word based on the context, thereby obtaining natural language text that conforms to the graph structure. The process is as follows: Figure 2 shown.

[0025] The process of using Graph Transformer to generate natural language text from knowledge graphs is as follows: First, the triples (s, r, o) in the knowledge graph are represented as the initial feature matrix H (0) , where s and o represent the subject and object nodes respectively, and r represents the relationship between them. The initial feature vector of a node is a numerical representation of the node attribute or semantic information, which can be generated by One-Hot encoding, embedding vectors, or directly normalizing the value; the initial feature vector of an edge represents the relationship attribute. After encoding, the initial feature vectors of nodes and edges are organized into the initial feature matrix H (0) , which is a matrix formed by stacking the initial eigenvectors of all nodes in rows, representing the initial state of the entire graph. (0) Input to Graph Transformer. Graph Transformer uses a multi-head attention mechanism to calculate the semantic relationship between a node and its neighboring nodes. The method for calculating the semantic relationship between a node and its neighboring nodes is: for each node v i , calculate the attention weight between it and its neighbor node N(i), the formula is:

[0026]

[0027] Where Q(Query) is the target node v i The query vector of the neighbor node; K (Key) is the key vector of the neighbor node; V (Value) is the value vector of the neighbor node; d kis the dimension of the key vector. The semantic relationship between a node and its neighboring nodes is represented by the calculated attention weights, so that Graph Transformer can learn the diverse relationships between nodes at each layer and aggregate these relationships into new node representations, thereby updating the feature representation of the nodes. After updating the node feature representation, the updated feature node representation is nonlinearly transformed through layer normalization and feedforward neural network to enhance Graph Transformer's ability to express complex relationships. Its process mainly consists of two stages:

[0028] a) First, the semantic relationship between the target node and its neighbor nodes is calculated through the multi-head attention mechanism to update the feature representation of the node:

[0029] H (l) =LayerNorm(H (l-1) +MultiHead(H (l-1) ))

[0030] The node feature H of the previous layer (l-1) As input, the multi-head attention mechanism MultiHead() extracts the interaction information between the target node and the neighboring nodes, and adds residual connections to retain the node's own information. Normalization is performed through LayerNorm() to ensure stable feature distribution and accelerate the training process. The final output H (l) Represents the node features of the lth layer.

[0031] b) Then use the multi-layer perceptron to analyze the output features H of the previous stage (l) Perform nonlinear mapping and feature enhancement to improve the expressiveness of output features:

[0032] H (l) '=LayerNorm(H (l) +MLP(H (l) ))

[0033] The output result H of the first stage (l) The input is sent to the multi-layer perceptron MLP() to learn higher-dimensional semantic representations and retain the original information through residual connections. Finally, LayerNorm() is used to normalize again to further improve the training stability of the model. This layer-by-layer update and information aggregation ensures that GraphTransformer can gradually integrate the relationship and semantic information in the knowledge graph to form a rich node representation in the task of the present invention.

[0034] Each layer l corresponds to a feature update of the network, starting from the initial feature matrix H (0) At the beginning, after layer-by-layer calculation, the output feature H of the Lth layer is finally obtained (L)(The above process is repeated L times, and H(l) is continuously iterated to H(L)), which represents the final semantic representation of the entire graph. After multiple layers of calculation and update, the final node representation H (L) Contains the complete semantic information extracted from the input knowledge graph. At this time, H (L) The decoder is input into the decoder, whose task is to use this semantic information to gradually generate natural language text. In the process of using this semantic information to gradually generate natural language text, the decoder predicts the probability distribution of generating the next word at each time step (a time step refers to each position in the generated sequence) based on the current context information:

[0035] P(w t |w <t ,H (L) )=softmax(W d H (L) )

[0036] Among them, W d represents the weight matrix of the decoder; w t represents the target word generated at time step t; w <t Represents all the words generated before time step t. The decoder adopts a step-by-step generation strategy, referring to the generated text and the semantic information stored in the GraphTransformer model in each step to generate the next word to ensure the coherence and accuracy of the generation process. In order to improve the quality and coherence of the generated text, the beam search algorithm is used to optimize the generation path. The beam search ensures that the generated natural language text is both semantically logical and high-quality in content and expression by retaining multiple high-probability candidate paths at each step and selecting the optimal path in the final stage.

[0037] In the natural language text generation task, sufficient supervised training is required to ensure that the GraphTransformer model has a good generation effect. In the model training stage of the present invention, the text data represented by the labeled triples is used for supervised learning to minimize the cross entropy loss function of the generated sequence:

[0038]

[0039] Through the back-propagation algorithm, the parameters of the Graph Transformer model are gradually updated to improve the semantic accuracy and fluency of the generated text, ensuring that the generated text is highly consistent with the structure of the knowledge graph.

[0040] After generating natural language text, it is also necessary to evaluate the quality of the generated text. If the quality does not meet the preset standards, the Graph Transformer model is retrained, and then the updated Graph Transformer model is used to generate natural language text; if the quality meets the preset standards, the next step of "large language model driven data enhancement" is executed. When evaluating the quality of the generated text, the present invention adopts the BLEU (Bilingual Evaluation Understudy) indicator, focusing on measuring the overlap between the generated text and the reference text on n-gram. BLEU can quantify the accuracy of the generated text and ensure that the output has high reliability in real-world applications. The core formula is as follows:

[0041]

[0042] Among them, p n Indicates the n-gram matching accuracy, w n is the weight, BP (brevity penalty) is used to penalize text that is too short, and N is the maximum length used to calculate n-gram matching. By using this indicator, the quality of the generated text can be further guaranteed, so that the final text fully conforms to the structure and content of the knowledge graph, which is convenient for subsequent work.

[0043] Step 2: Perform data enhancement driven by a large language model on the natural language text generated in step 1 to generate enhanced text

[0044] Large Language Models (LLMs) are language models that are trained on a large scale to understand and generate human language. Applications of LLMs include, but are not limited to, text generation, language understanding, and translation. Text data augmentation refers to the use of various technical means to expand existing text datasets to improve the performance and generalization ability of machine learning models. By using LLMs, prompt engineering techniques can be used to generate large amounts of new data. Prompt engineering refers to designing appropriate prompts to guide LLMs to generate text with desired features. These prompts can be in the form of questions, commands, conditions, etc., to guide LLMs to generate text data that meets specific criteria.

[0045] The present invention performs data enhancement on the natural language text data generated from the knowledge graph through a dual enhancement strategy at the context level and the entity level to ensure that the generated text is in line with reality in terms of structure and content. First, preliminary enhancement at the context level is adopted, that is, different text expressions are generated by adjusting sentence length, vocabulary usage, clause structure and text style, increasing the diversity of the text while maintaining the triple structure. Then, enhancement is performed at the entity level, using LLM to replace the entities in the original sentences of the natural language text with other entities of the same type, enriching the diversity of the data and ensuring that the content conforms to the real logic. The process is as follows Figure 3 shown.

[0046] In context-level data augmentation, we designed adjustments in four main dimensions, namely sentence length, vocabulary usage, clause structure, and text style, to achieve diversified expansion of text expression. For a given text T, the new text T' generated by context-level data augmentation can be expressed as T'=f context (T,p), where p represents the specific application of the enhancement strategy. In sentence length adjustment, the sentence length is extended or shortened by adding or deleting content to generate different T's to adapt to different contextual scenarios and task requirements. The adjustment of vocabulary usage involves the replacement of vocabulary in the text, such as replacing antecedents, adjectives or prepositions, to ensure semantic diversity without affecting the core meaning of the text. The optimization of clause structure allows the type and position of clauses to be adjusted to generate texts with different structural complexity and increase the richness of the data. In addition, by changing the presentation style of the text, texts that adapt to different scenarios can be generated, enhancing the flexibility of the model in practical applications.

[0047] After the initial enhancement at the context level, the text needs to be further processed, that is, using LLM to implement entity replacement to increase the diversity of the text and achieve entity-level data enhancement. Given the entity set E contained in the text T', the generated enhanced text T" can be expressed as T" = f entity (T', E'), where E' is the entity set to be replaced. Different from the traditional entity replacement method, the present invention uses the conditional probability P(e' i |context(T')) to select the appropriate replacement entity e' i , that is, the new entity, context(T') represents the context of T', ensuring that the new entity is semantically consistent with the text context. This strategy can generate new text containing entities that do not appear in the original data (i.e., text T'), thereby expanding the diversity of the dataset and enabling the model to have higher generalization and adaptability.

[0048] In addition, in order to deal with potential semantic drift and inconsistency in data enhancement, a supervision mechanism is introduced in the data enhancement process to ensure the logical and semantic coherence of the generated text. By limiting entity replacement to semantically related or similar entities, new entities can be introduced without destroying the original semantic relationship, effectively avoiding the irrationality of generated content.

[0049] Step 3: Extract triples from the enhanced text and build a new knowledge graph

[0050] Fine-tuning refers to additional training on a pre-trained model using a new dataset to adapt the model to new tasks or new data distributions. In the field of natural language processing, fine-tuning usually refers to fine-tuning a pre-trained language model to adapt it to specific text classification, text generation, or other related tasks. The core idea of ​​fine-tuning is to use the rich language representations learned by the pre-trained model to accelerate the learning of new tasks, and by fine-tuning on new task data, the model can better understand and generalize new data. By fine-tuning the pre-trained language model, the model can have better semantic understanding capabilities, thereby more effectively building a knowledge graph.

[0051] The present invention generates triples of enhanced text through fine-tuning technology, and generates new knowledge graphs based on these triples. By fine-tuning the BERT model on the OntoNotes 5.0 and TACRED datasets, it can adapt to entity recognition and relationship extraction tasks. The fine-tuned model can accurately identify entities and their categories in the text, and divide them into multiple triples by category. Then, a new knowledge graph is generated based on each triple. The new knowledge graph is consistent with the original knowledge graph in structure, but it achieves richness and diversity in content. The process is as follows Figure 4 shown.

[0052] In the process of building a new knowledge graph, the pre-trained BERT model is selected as the fine-tuning model. The model performs well in a variety of natural language processing tasks, especially its bidirectional context understanding ability, making it suitable for entity recognition tasks and relationship extraction tasks.

[0053] For the entity recognition task, the OntoNotes 5.0 dataset is selected for fine-tuning. In this dataset, the data is annotated into different entity types, including names of people, places, and organizations, and each training data is organized into input text and its corresponding label. Assume that the input text is T, and its corresponding entity label set is E = {e1, e2, …, e m}, the goal of fine-tuning is to optimize the parameters of the BERT model so that the model can effectively identify these entities from the input text. The training objective function of the model can be expressed as:

[0054]

[0055] Among them, y t is the label of the tth word in the given text T, P(y t |T) is the model's predicted probability for the label. Through the back-propagation algorithm, the model parameters are updated to minimize this objective function, thereby improving the accuracy of the model in the entity recognition task.

[0056] After completing model fine-tuning, the enhanced text data is input into the fine-tuned BERT model for entity recognition. The input of the model is the enhanced text dataset D, and its output is the entity set E' in the text. The number of entities is m, and each entity e' i The category C(e' i ). The output of entity recognition can be expressed as:

[0057] E”={(e'1,C(e'1)),(e'2,C(e'2)),…,(e' m' ,C(e' m' ))}

[0058] For the relationship extraction task, we select the TACRED dataset for fine-tuning. After fine-tuning, we perform context analysis on the identified entities and determine the relationship between the entities. So far, the triples identified by the fine-tuned model can be expressed as:

[0059] R={(s,r,o)|s∈E”,r∈Relations,o∈E”}

[0060] Among them, s is the subject entity (i.e., subject node), o is the object entity (i.e., object node), r is the relationship between them, and Relations represents the entity relationship set. In the process of identifying triples, the model optimizes the generation of each triple based on the contextual information in the text, the category of the entity, and the relationship knowledge learned during training. Through sequence labeling and classification methods, the model can effectively extract the formal data of triples (s, r, o). These triplets must not only accurately reflect the information in the enhanced text, but also ensure that they are logically consistent with the actual scenario.

[0061] Once all triples are generated, they can be divided into multiple sets according to the categories of entities in the triples. Each set is used to generate the corresponding new knowledge graph G'=(V',E'), where V' represents the node set (i.e., entity) in the new knowledge graph and E' represents the edge set (i.e., relationship), which can be expressed as:

[0062] G'=(V',E')=({s1,s2,…,s n},{(s i ,rj ,o k )|(s i ,r j ,o k )∈R})

[0063] Ultimately, the generated new graph is consistent with the original knowledge graph in structure, with increased diversity and richness in content, and is consistent with reality.

[0064] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for generating relational data by means of a knowledge graph, characterized in that: The following steps are involved: Step 1: Use Graph Transformer to generate natural language text from the knowledge graph. Graph Transformer is a graph neural network model based on the graph attention mechanism. It uses the attention mechanism to transfer and aggregate information in the graph structure and convert this information into natural language description. Step 2: Perform data enhancement driven by a large language model on the natural language text generated in step 1 to generate enhanced text; Among them, the natural language text data generated from the knowledge graph is enhanced through a dual enhancement strategy at the context level and the entity level to ensure that the generated text is consistent with reality in terms of structure and content. First, preliminary enhancement at the context level is adopted, that is, by adjusting the sentence length, vocabulary usage, clause structure and text style to generate different text expressions, while maintaining the triple structure and increasing the diversity of the text; then enhancement is performed at the entity level, using LLM to replace the entities in the original sentences of the natural language text with other entities of the same type, enriching the diversity of the data and ensuring that the content conforms to the real logic; Step 3: Extract triples from the enhanced text and construct a new knowledge graph; Among them, the triples of enhanced text are generated through fine-tuning technology, and a new knowledge graph is generated based on the generated triples; among them, by fine-tuning the BERT model on the OntoNotes 5.0 and TACRED datasets, the BERT model is made adaptable to entity recognition and relationship extraction tasks. The fine-tuned BERT model can recognize entities and their categories in the text, and divide them into multiple triples according to categories; and then a new knowledge graph is generated based on each triple. The new knowledge graph is consistent with the original knowledge graph in structure, but achieves richness and diversity in content.

2. The method according to claim 1, characterized in that The process of using Graph Transformer to generate natural language text from the knowledge graph in step 1 is as follows: the triples consisting of subject, relation, and object in the original knowledge graph are abstracted into node and edge representations and input into GraphTransformer for processing; GraphTransformer uses a multi-head attention mechanism to identify the semantic relationship and importance between each node and its neighboring nodes, and updates the node representation; Then, nonlinear transformation is performed through normalization and feedforward neural network to enhance the expressiveness and stability of the Graph Transformer model. The updated node representation is input into the decoder, and the decoder gradually predicts the next word based on the context, thereby obtaining natural language text that conforms to the graph structure.

3. The method according to claim 2, characterized in that The process of using Graph Transformer to generate natural language text from the knowledge graph in step 1 is as follows: First, the triples (s, r, o) in the knowledge graph are represented as the initial feature matrix H (0) , where s and o represent the subject and object nodes respectively, and r represents the relationship between them; the initial feature vector of the node is the numerical representation of the node attribute or semantic information, which can be generated by One-Hot encoding, embedding vector or direct normalization; the initial feature vector of the edge represents the relationship attribute; the initial feature vectors of the node and edge are encoded and organized into the initial feature matrix H (0) , that is, the matrix formed by stacking the initial eigenvectors of all nodes in rows, represents the initial state of the entire graph; then the initial eigenvector matrix H (0) Input into Graph Transformer; Graph Transformer calculates the semantic relationship between a node and its neighboring nodes through a multi-head attention mechanism; the method for calculating the semantic relationship between a node and its neighboring nodes is: for each node v i , calculate the attention weight between it and its neighbor node N(i), the formula is: Where Q(Query) is the target node v i The query vector of the neighbor node; K (Key) is the key vector of the neighbor node; V (Value) is the value vector of the neighbor node; d k is the dimension of the key vector; the semantic relationship between a node and its neighboring nodes is represented by the calculated attention weights, and then the Graph Transformer learns the diverse relationships between nodes at each layer and aggregates these relationships into new node representations, thereby updating the feature representations of the nodes; after updating the node feature representations, the updated feature node representations are nonlinearly transformed through layer normalization and feedforward neural networks to enhance the Graph Transformer's ability to express complex relationships; after multiple layers of calculation and updating, the node representation H containing the complete semantic information extracted from the input knowledge graph is finally obtained. (L) ; H (L) Input decoder, decoder uses H (L) The semantic information contained in the code is gradually used to generate natural language text.

4. The method according to claim 3, characterized in that Graph Transformer learns the diverse relationships between nodes at each layer and aggregates these relationships into new node representations, thereby updating the feature representations of the nodes. After updating the node feature representations, the updated feature node representations are nonlinearly transformed through layer normalization and feedforward neural networks to enhance Graph Transformer's ability to express complex relationships. After multiple layers of calculation and updating, the node representation H containing the complete semantic information extracted from the input knowledge graph is finally obtained. (L) The process consists of two stages composition: a) First, the semantic relationship between the target node and its neighbor nodes is calculated through the multi-head attention mechanism to update the feature representation of the node: H (l) =LayerNorm(H (l-1) +MultiHead(H (l-1) )) The node feature H of the previous layer (l-1) As input, the interaction information between the target node and the neighboring nodes is extracted through the multi-head attention mechanism MultiHead(), and residual connections are added to retain the node's own information; normalization is performed through LayerNorm() to ensure the stability of feature distribution and accelerate the training process; the final output H (l) Represents the node features of the lth layer; b) Then use the multi-layer perceptron to analyze the output features H of the previous stage (l) Perform nonlinear mapping and feature enhancement to improve the expressiveness of output features: H (l) '=LayerNorm(H (l) +MLP(H (l) )) The output result H of the first stage (l) The data is input into the multi-layer perceptron MLP() to learn higher-dimensional semantic representations and retain the original information through residual connections. Finally, it is normalized again using LayerNorm() to further improve the training stability of the model. Each layer l corresponds to a feature update of the network, starting from the initial feature matrix H (0) At the beginning, after layer-by-layer calculation, the output feature H of the Lth layer is finally obtained (L) , represents the final semantic representation of the entire graph; after multiple layers of calculation and updating, the final node representation H (L) Contains complete semantic information extracted from the input knowledge graph.

5. The method according to claim 3, characterized in that In the process of the decoder gradually generating natural language text using semantic information, the decoder predicts the probability distribution of generating the next word at each time step based on the current context information: P(w t |w <t ,H (L) )=softmax(W d H (L) ) Among them, W d represents the weight matrix of the decoder; w t represents the target word generated at time step t; w <t Represents all the words generated before time step t; the decoder adopts a step-by-step generation strategy, and generates the next word at each step by referring to the generated text and the semantic information stored in the GraphTransformer model. The beam search algorithm is used to optimize the generation path. The beam search retains multiple high-probability candidate paths at each step and selects the optimal path in the final stage.

6. The method according to claim 3, characterized in that In the process of natural language text generation, the GraphTransformer model is also trained through the model. In the model training stage, the text data represented by the labeled triples is used for supervised learning to minimize the cross entropy loss function of the generated sequence: Through the back-propagation algorithm, the parameters of the Graph Transformer model are gradually updated to improve the semantic accuracy and fluency of the generated text.

7. The method according to claim 6, characterized in that After generating the natural language text, the quality of the generated text is also evaluated. If the quality does not meet the preset standard, the Graph Transformer model is retrained and then the updated Graph Transformer model is used to generate the natural language text. If the quality meets the preset standard, step 2 is executed. When evaluating the quality of the generated text, the BLEU indicator is used to measure the overlap between the generated text and the reference text on n-gram. The calculation formula of the BLEU indicator is as follows: Among them, p n Indicates the n-gram matching accuracy, w n is the weight, BP is used to penalize texts that are too short, and N is the maximum length used to calculate the accuracy of n-gram matching.

8. The method according to claim 3, characterized in that In step 2, in the context-level data enhancement, adjustments in four main dimensions, namely sentence length, vocabulary usage, clause structure and text style, are designed, i.e., four enhancement strategies, to achieve diversified expansion of text expression; for a given text T, the new text T' generated by the context-level data enhancement is expressed as T'=f context (T,p), where p represents the enhancement strategy; in sentence length adjustment, the sentence length is extended or shortened by adding or deleting content to generate different T's to adapt to different contextual scenarios and task requirements; vocabulary usage adjustment involves the replacement of vocabulary in the text; the optimization of clause structure allows the type and position of clauses to be adjusted to generate texts with different structural complexity; in addition, by changing the presentation style of the text, texts that adapt to different scenarios can be generated.

9. The method according to claim 8, characterized in that After the initial enhancement at the context level, the text after the initial enhancement is further processed, that is, LLM is used to implement entity replacement to increase the diversity of the text and realize entity-level data enhancement; given the entity set E contained in the text T', the generated enhanced text T ″ Represented as T ″ =f entity (T', E'), where E' is the set of entities to be replaced; by conditional probability P(e' i |context(T')) to select the appropriate replacement entity e' i , that is, the new entity, context(T') represents the context of T', ensuring that the new entity is semantically consistent with the text context; this strategy can generate new text containing entities that do not appear in the text T', thereby expanding the diversity of the dataset.

10. The method according to claim 9, characterized in that In step 3, in the process of building a new knowledge graph, the pre-trained BERT model is selected as the fine-tuning model to perform entity recognition tasks and relationship extraction tasks; For the entity recognition task, the OntoNotes 5.0 dataset is selected for fine-tuning. In the OntoNotes 5.0 dataset, the data is labeled with different entity types, and each training data is organized into an input text and its corresponding label. Assuming that the input text is T, its corresponding entity label set is E = {e1, e2, …, e m The goal of fine-tuning is to optimize the parameters of the BERT model so that the BERT model can recognize these entities from the input text. The training objective function of the BERT model is expressed as: Among them, y t is the label of the tth word in the given text T, P(y t |T) is the predicted probability of the label by the BERT model; through the back-propagation algorithm, the BERT model parameters are updated to minimize this objective function; After completing model fine-tuning, the enhanced text data is input into the fine-tuned BERT model for entity recognition; the input of the BERT model is the enhanced text dataset D, and its output is the entity set E' in the text, the number of entities is m, and each entity e' i The category C(e' i ); the output of entity recognition is expressed as: And”={(e'1,C(e'1)),(e'2,C(e'2)),…,(e' m' ,There is' m' ))} For the relationship extraction task, the TACRED dataset is selected for fine-tuning. After fine-tuning, the context analysis of the identified entities is performed and the relationship between the entities is determined. At this point, the triples identified by the fine-tuned model are expressed as: R={(s,r,o)|s∈E”,r∈Relations,o∈E”} Among them, s is the subject entity, o is the object entity, r is the relationship between them, and Relations represents the entity relationship set. In the process of identifying triples, the model optimizes the generation of each triple based on the contextual information in the text, the category of the entity, and the relationship knowledge learned during the training process; through sequence labeling and classification methods, the model can effectively extract the formal data of the triple (s, r, o); Once all triples are generated, they are divided into multiple sets according to the categories of entities in the triples. Each set is used to generate the corresponding new knowledge graph G'=(V',E'), where V' represents the node set (i.e., entity) in the new knowledge graph, and E' represents the edge set (i.e., relationship). The new knowledge graph is represented as: G'=(V',E')=({s1,s2,…,s n },{(s i ,r j ,o k )|(s i ,r j ,o k )∈R}) Ultimately, the generated new graph is consistent with the original knowledge graph in structure, with increased diversity and richness in content, and is consistent with reality.

Images