A method for generating relational data using knowledge graphs

Abstract

This invention relates to a method for generating relational data using knowledge graphs, belonging to the field of artificial intelligence technology. By using a Graph Transformer, leveraging multi-head attention mechanisms and neighborhood aggregation in a feedforward neural network, this invention effectively captures the semantic relationships and importance between nodes, thus preserving the features of the original structure in the newly generated graph and ensuring consistency between the new and original graph structures. Furthermore, by using a Large Language Model (LLM) combined with cue engineering for data augmentation, augmentation strategies are applied at the context and entity levels. While maintaining the original triplet structure, replacement strategies ensure that the augmented text content is consistent with real-world logic. Fine-tuning the pre-trained model to adapt to entity recognition and relation extraction tasks further improves the reliability and practical application value of the generated content.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to a method for generating relational data using knowledge graphs. Background Technology

[0002] Various graph generation techniques exist in the current field of knowledge graph generation, including rule-based generation, graph embedding methods, and deep learning models. However, due to the limitations of their respective principles, they face a series of problems in the graph generation process. Rule-based generation methods rely on predefined rules, lacking flexibility, resulting in generated graphs that are easily simplified or deformed in structure, and making it difficult to maintain the complex relationships of the original graph. Graph embedding methods generate new graphs by mapping the graph structure to a low-dimensional space, but in this process, they often ignore the detailed relationships of the graph, resulting in a graph structure that is often inconsistent with the original graph, and easily introducing fictitious nodes or relationships. Deep learning-based generative models (such as GANs and variational autoencoders) rely on data distribution to generate nodes and relationships, but due to the randomness of the generated content, it is difficult to ensure that the new graph structure is consistent with the original graph, and it may produce fictitious relationships or entities that do not conform to reality. These problems make it difficult to guarantee the consistency and reliability of the generated knowledge graphs in practical applications. These problems limit the usability of the new knowledge graphs. Summary of the Invention

[0003] (a) Technical problems to be solved

[0004] The technical problems to be solved by the present invention are: 1. To solve the problem that the new maps generated in the prior art have significant structural deviations from the original maps, making it difficult to maintain consistency; 2. To solve the problem that the content of the maps generated in the prior art often does not match the real scene, lacking authenticity and usability.

[0005] (II) Technical Solution

[0006] To address the aforementioned technical problems, this invention provides a method for generating relational data using knowledge graphs, comprising the following steps:

[0007] Step 1: Use Graph Transformer to generate natural language text from the knowledge graph; Graph Transformer is a graph neural network model based on graph attention mechanism. It uses attention mechanism to transfer and aggregate information in the graph structure and transform this information into natural language description.

[0008] Step 2: Perform large language model-driven data augmentation on the natural language text generated in Step 1 to generate augmented text;

[0009] Specifically, a dual enhancement strategy at the context and entity levels is employed to augment natural language text data generated from knowledge graphs, ensuring that the generated text conforms to reality in terms of structure and content. First, context-level enhancement is used to generate different text expressions by adjusting sentence length, vocabulary usage, clause structure, and text style, increasing text diversity while maintaining the triplet structure. Then, entity-level enhancement is performed by using LLM to replace entities in the original natural language text with other entities of the same type, enriching data diversity and ensuring that the content conforms to real-world logic.

[0010] Step 3: Extract triples from the enhanced text and construct a new knowledge graph;

[0011] This process involves generating triples for enhanced text through fine-tuning techniques, and then creating a new knowledge graph based on these triples. Specifically, by fine-tuning the BERT model on the OntoNotes 5.0 and TACRED datasets, the BERT model is adapted to entity recognition and relation extraction tasks. The fine-tuned BERT model can identify entities and their categories in the text and categorize them into multiple triples. A new knowledge graph is then generated based on each triple. The new knowledge graph maintains the same structure as the original knowledge graph but achieves richness and diversity in its content.

[0012] (III) Beneficial Effects

[0013] This invention proposes a method for generating relational data using knowledge graphs. The implementation of this method can achieve the following effects:

[0014] 1. Ensure the generated graph structure is similar to the original graph: By using Graph Transformer, multi-head attention mechanism and neighborhood aggregation of feedforward neural network can be used to effectively capture the semantic association and importance between nodes, so as to retain the features of the original structure in the newly generated graph and ensure that the newly generated graph structure is consistent with the original graph structure.

[0015] 2. Ensuring the generated graph content aligns with real-world scenarios: Data augmentation is performed using a Large Language Model (LLM) combined with cue engineering. Augmentation strategies are applied at both the context and entity levels. While maintaining the original triple structure, replacement strategies ensure the augmented text content remains consistent with real-world logic. The pre-trained model is fine-tuned to adapt to entity recognition and relation extraction tasks, further improving the reliability and real-world application value of the generated content. Attached Figure Description

[0016] Figure 1 This is a general framework diagram of the method of the present invention;

[0017] Figure 2This is a flowchart of the natural text statements generated in the knowledge graph in the method of the present invention;

[0018] Figure 3 This is a flowchart illustrating the data augmentation process using a large language model in the method of this invention.

[0019] Figure 4 This is a flowchart of the method for extracting triples and constructing a knowledge graph in this invention. Detailed Implementation

[0020] To make the objectives, contents, and advantages of the present invention clearer, the specific embodiments of the present invention will be described in further detail below with reference to the accompanying drawings and examples.

[0021] This invention proposes a method for generating relational data using knowledge graphs. Specifically, it generates a new knowledge graph based on an existing one, ensuring that the generated graph's structure is similar to the original graph and that its content is realistic, containing no fictitious or unreasonable information. This invention aims to overcome the limitations of current knowledge graph generation technologies, providing more reliable and practical data support for knowledge graph applications, promoting the widespread application of knowledge graph technology across various industries, and ultimately driving the development of intelligent decision-making and services.

[0022] The overall framework of this method is as follows: Figure 1 As shown.

[0023] Step 1: Generating Natural Language Text from Knowledge Graphs Using Graph Transformer. Graph Transformer is a graph neural network model based on graph attention mechanisms, capable of extracting accurate natural language text from knowledge graphs. Unlike traditional graph neural networks, Graph Transformer uses attention mechanisms instead of simple summation operations in the neighborhood aggregation process, allowing the model to more flexibly capture the relationships and importance between nodes. The Graph Transformer model also includes components such as normalization and MLP feedforward neural networks to enhance its expressive power and performance. The main advantage of Graph Transformer lies in its ability to effectively transfer and aggregate information within the graph structure using attention mechanisms, thereby better capturing the complex relationships and importance between entities and transforming this information into natural language descriptions.

[0024] The process of generating natural language text from a knowledge graph using Graph Transformer is as follows: The triples (composed of subject, relation, and object) in the original knowledge graph are abstracted into node and edge representations, and then input into Graph Transformer for processing. Graph Transformer uses a multi-head attention mechanism to identify the semantic relationships and importance between each node and its neighbors, and updates the node representations accordingly. Then, normalization and a feedforward neural network are used for nonlinear transformations to enhance the expressive power and stability of the Graph Transformer model. The updated node representations are input into the decoder, which predicts the next word progressively based on the context, thus obtaining natural language text that conforms to the graph structure. The process is as follows: Figure 2 As shown.

[0025] The process of generating natural language text from a knowledge graph using Graph Transformer is as follows: First, the triples (s, r, o) in the knowledge graph are represented as the initial feature matrix H. (0) Here, s and o represent the subject and object nodes, respectively, while r represents the relationship between them. The initial feature vector of a node is a numerical representation of its attributes or semantic information, which can be generated through One-Hot encoding, embedding vectors, or direct normalization of values; the initial feature vector of an edge represents the relation attributes. After encoding, the initial feature vectors of nodes and edges are arranged into an initial feature matrix H. (0) H is a matrix formed by stacking the initial feature vectors of all nodes row-wise, representing the initial state of the entire graph. This initial feature matrix H is then used to... (0) The input is fed into the Graph Transformer. The Graph Transformer uses a multi-head attention mechanism to compute the semantic relationships between nodes and their neighbors. The method for computing the semantic relationships between nodes and their neighbors is as follows: for each node v... i Calculate its attention weight with its neighbor node N(i), using the following formula:

[0026]

[0027] Where Q (Query) is the target node v i The query vector; K (Key) is the key vector of neighboring nodes; V (Value) is the value vector of neighboring nodes; d kThis refers to the dimension of the key vector. The semantic relationship between a node and its neighboring nodes is represented by computed attention weights. This allows the Graph Transformer to learn diverse relationships between nodes at each layer and aggregate these relationships into a new node representation, thereby updating the node's feature representation. After updating the node feature representation, the updated feature node representation undergoes a non-linear transformation through layer normalization and a feedforward neural network to enhance the Graph Transformer's ability to express complex relationships. The process mainly consists of two stages:

[0028] a) First, the semantic relationship between the target node and its neighboring nodes is calculated using a multi-head attention mechanism, thereby updating the node's feature representation:

[0029] H (l) =LayerNorm(H (l-1) +MultiHead(H (l-1) ))

[0030] The node features H of the previous layer (l-1) As input, the target node's interaction information with its neighbors is extracted using the MultiHead() multi-head attention mechanism, and residual connections are added to preserve the node's own information. LayerNorm() is then used for normalization to ensure stable feature distribution and accelerate the training process. The final output H... (l) This represents the node characteristics of the l-th layer.

[0031] b) Then, the multilayer perceptron is used to analyze the output features H of the previous stage. (l) Nonlinear mapping and feature enhancement are performed to improve the expressive power of the output features:

[0032] H (l) =LayerNorm(H (l) +MLP(H (l) ))

[0033] The output result H of the first stage (l) The data is fed into a Multilayer Perceptron (MLP()) to learn higher-dimensional semantic representations, and the original information is preserved through residual connections. Finally, LayerNorm() is used for further normalization to improve the training stability of the model. This layer-by-layer update and information aggregation ensures that GraphTransformer can progressively integrate relational and semantic information from the knowledge graph to form rich node representations in the task of this invention.

[0034] Each layer l corresponds to one feature update of the network, starting from the initial feature matrix H. (0) Initially, the output feature H of the Lth layer is obtained through layer-by-layer calculation. (L)(The above process is repeated L times, with H(l) iterating continuously to H(L)), representing the final semantic representation of the entire graph. After multiple layers of computation and updates, the final node representation H is obtained. (L) It contains complete semantic information extracted from the input knowledge graph. At this point, H (L) The information is input into the decoder, whose task is to use this semantic information to progressively generate natural language text. In this process, the decoder predicts the probability distribution of generating the next word at each time step (a time step refers to each position in the generated sequence) based on the current context information.

[0035] P(w t |w <t H (L) = softmax(W) d H (L) )

[0036] Among them, W d w represents the weight matrix of the decoder. t This represents the target word generated at time step t; w <t This represents all words generated before time step t. The decoder employs a stepwise generation strategy, referencing the already generated text and semantic information stored in the GraphTransformer model at each step to generate the next word, ensuring the coherence and accuracy of the generation process. To improve the quality and coherence of the generated text, a beam search algorithm is used to optimize the generation path. Beam search retains multiple high-probability candidate paths at each step and selects the optimal path in the final stage, ensuring that the generated natural language text is both semantically logical and maintains high quality in content and expression.

[0037] In natural language text generation tasks, sufficiently supervised training is necessary to ensure that the GraphTransformer model achieves good generation performance. In the model training phase of this invention, labeled triplet-represented text data is used for supervised learning to minimize the cross-entropy loss function of the generated sequences.

[0038]

[0039] Through the backpropagation algorithm, the parameters of the Graph Transformer model are updated step by step to improve the semantic accuracy and fluency of the generated text, ensuring that the generated text closely matches the structure of the knowledge graph.

[0040] After generating natural language text, the quality of the generated text needs to be evaluated. If the quality does not meet the preset standards, the Graph Transformer model is retrained, and then the updated Graph Transformer model is used to generate natural language text again. If the quality meets the preset standards, the next step, "data augmentation driven by a large language model," is executed. When evaluating the quality of the generated text, this invention uses the BLEU (Bilingual Evaluation Understudy) metric, which focuses on measuring the degree of overlap between the generated text and the reference text in terms of n-grams. BLEU can quantify the accuracy of the generated text, ensuring that the output has high reliability in real-world applications. Its core formula is as follows:

[0041]

[0042] Where, p n w represents the accuracy of n-gram matching. n BP (brevitypenalty) is used to penalize excessively short text, and N is the maximum length used to calculate n-gram matches. Using this metric further ensures the quality of the generated text, guaranteeing that the final text fully conforms to the structure and content of the knowledge graph, facilitating its use in subsequent work.

[0043] Step 2: Perform large language model-driven data augmentation on the natural language text generated in Step 1 to generate augmented text.

[0044] Large Language Models (LLMs) are language models that have been trained on a large scale to understand and generate human language. Applications of LLMs include, but are not limited to, text generation, language understanding, and translation. Text data augmentation refers to using various techniques to expand existing text datasets to improve the performance and generalization ability of machine learning models. By using LLMs, prompt engineering techniques can be used to generate large amounts of new data. Prompt engineering involves designing appropriate prompts to guide LLM in generating text with desired features. These prompts can take the form of questions, commands, conditions, etc., guiding LLM to generate text data that conforms to specific criteria.

[0045] This invention employs a dual enhancement strategy at the context and entity levels to augment natural language text data generated from knowledge graphs, ensuring that the generated text conforms to reality in both structure and content. First, context-level enhancement is used to generate different text expressions by adjusting sentence length, vocabulary usage, clause structure, and text style, increasing text diversity while maintaining the triple structure. Then, entity-level enhancement is performed, using LLM to replace entities in the original natural language text with other entities of the same type, enriching data diversity and ensuring the content conforms to real-world logic. The process is as follows: Figure 3 As shown.

[0046] In context-level data augmentation, adjustments were made to four main dimensions: sentence length, vocabulary usage, clause structure, and text style—that is, four augmentation strategies—to achieve diversified expansion of text expression. For a given text T, the new text T' generated by context-level data augmentation can be represented as T' = f context (T,p), where p represents the specific application of the enhancement strategy. Sentence length adjustment involves expanding or shortening sentences by adding or deleting content, generating different T' values ​​to adapt to different contexts and task requirements. Lexical usage adjustment involves replacing words in the text, such as changing antecedents, adjectives, or prepositions, to ensure semantic diversity without affecting the core meaning of the text. Optimization of clause structure allows for adjustments to the type and position of clauses to generate texts with varying structural complexity, increasing data richness. Furthermore, by changing the text presentation style, text adapted to different scenarios can be generated, enhancing the model's flexibility in practical applications.

[0047] After initial context-level augmentation, further processing is needed on the augmented text. This involves using LLM (Limited Module Management) to implement entity replacement, increasing text diversity and achieving entity-level data augmentation. Given the entity set E contained in text T', the generated augmented text T″ can be represented as T″ = f entity (T', E'), where E' is the set of entities to be replaced. Unlike traditional entity replacement methods, this invention uses conditional probability P(e') i Use `context(T')` to select the appropriate replacement entity `e'`. i The new entity is defined as `context(T')`, which represents the context of `T'`, ensuring semantic consistency between the new entity and the text context. This strategy can generate new text containing entities that do not appear in the original data (i.e., text `T'`), thereby expanding the diversity of the dataset and enabling the model to have higher generalization ability and adaptability.

[0048] Furthermore, to address potential semantic shifts and inconsistencies in data augmentation, a supervisory mechanism is introduced during the data augmentation process to ensure the logical and semantic coherence of the generated text. By restricting entity replacement to semantically related or similar entities, new entities can be introduced without disrupting the original semantic relationships, effectively avoiding irrationality in the generated content.

[0049] Step 3: Extract triples from the enhanced text and construct a new knowledge graph.

[0050] Fine-tuning refers to further training a pre-trained model using new datasets to adapt it to new tasks or data distributions. In natural language processing, fine-tuning typically involves tweaking a pre-trained language model to suit specific tasks such as text classification, text generation, or other related tasks. The core idea of ​​fine-tuning is to leverage the rich language representations already learned by the pre-trained model to accelerate learning for new tasks. By fine-tuning on new task data, the model can better understand and generalize new data. Fine-tuning a pre-trained language model can enhance its semantic understanding capabilities, thereby enabling more effective knowledge graph construction.

[0051] This invention generates enhanced text triples through fine-tuning techniques and then generates a new knowledge graph based on these triples. By fine-tuning the BERT model on the OntoNotes 5.0 and TACRED datasets to adapt it to entity recognition and relation extraction tasks, the fine-tuned model can accurately identify entities and their categories in the text and categorize them into multiple triples. A new knowledge graph is then generated based on each triple. The new knowledge graph maintains the same structure as the original knowledge graph but achieves richness and diversity in content. The process is as follows: Figure 4 As shown.

[0052] In the process of constructing the new knowledge graph, the pre-trained BERT model was selected as the fine-tuning model. This model performs well in a variety of natural language processing tasks, especially its bidirectional contextual understanding ability, which makes it suitable for entity recognition and relation extraction tasks.

[0053] For the entity recognition task, the OntoNotes 5.0 dataset was selected for fine-tuning. In this dataset, data is labeled with different entity types, including names of people, places, and organizations. Each training data point is organized as input text and its corresponding label. Assuming the input text is T, its corresponding entity label set is E = {e1, e2, ..., e...} m The goal of fine-tuning is to optimize the parameters of the BERT model so that it can effectively identify these entities from the input text. The model's training objective function can be expressed as:

[0054]

[0055] Among them, y t Let P(y) be the label of the t-th word in a given text T. t |T) represents the model's predicted probability for that label. The model parameters are updated using backpropagation to minimize this objective function, thereby improving the model's accuracy in entity recognition tasks.

[0056] After fine-tuning the model, the enhanced text data is input into the fine-tuned BERT model for entity recognition. The model's input is the enhanced text dataset D, and its output is the set of entities E' in the text, with m entities and each entity e'... i Its category C(e') is labeled i The output of entity recognition can be represented as:

[0057] E”={(e'1,C(e'1)),(e'2,C(e'2)),…,(e' m' ,C(e' m' ))}

[0058] For the relation extraction task, the TACRED dataset was selected for fine-tuning. After fine-tuning, context analysis was performed on the identified entities to determine the relationships between them. Thus, the triples identified by the fine-tuned model can be represented as:

[0059] R={(s,r,o)|s∈E”,r∈Relations,o∈E”}

[0060] Here, 's' is the subject entity (i.e., the subject node), 'o' is the object entity (i.e., the object node), 'r' is the relationship between them, and 'Relations' represents the set of entity relations. During triple recognition, the model optimizes the generation of each triple based on contextual information in the text, entity categories, and relational knowledge learned during training. Through sequence labeling and classification methods, the model can effectively extract triple data in the form of (s, r, o). These triples must not only accurately reflect the information in the enhanced text but also ensure that they logically conform to the actual scenario.

[0061] Once all triples are generated, they can be divided into multiple sets based on the categories of the entities within them. Each set is used to generate a corresponding new knowledge graph G' = (V', E'), where V' represents the set of nodes (i.e., entities) in the new knowledge graph, and E' represents the set of edges (i.e., relations). This can be represented as:

[0062] G'=(V',E')=({s1,s2,…,s n},{(s i ,rj ,o k )|(s i ,r j ,o k )∈R})

[0063] Ultimately, the generated new graph maintains structural consistency with the original knowledge graph, while adding diversity and richness in content, and conforms to reality.

[0064] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the technical principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for generating relational data using knowledge graphs, characterized in that, Includes the following steps: Step 1: Use Graph Transformer to generate natural language text from the knowledge graph; Graph Transformer is a graph neural network model based on graph attention mechanism. It uses attention mechanism to transfer and aggregate information in the graph structure and transform this information into natural language description. Step 2: Perform data augmentation driven by a large language model on the natural language text generated in Step 1 to generate augmented text; Specifically, a dual enhancement strategy at the context and entity levels is employed to augment natural language text data generated from knowledge graphs, ensuring that the generated text conforms to reality in terms of structure and content. First, context-level enhancement is used to generate different text expressions by adjusting sentence length, vocabulary usage, clause structure, and text style, increasing text diversity while maintaining the triplet structure. Then, entity-level enhancement is performed by using LLM to replace entities in the original natural language text with other entities of the same type, enriching data diversity and ensuring that the content conforms to real-world logic. Step 3: Extract triples from the enhanced text and construct a new knowledge graph; This process involves generating triples for enhanced text through fine-tuning techniques, and then creating a new knowledge graph based on these triples. Specifically, by fine-tuning the BERT model on the OntoNotes 5.0 and TACRED datasets, the BERT model is adapted to entity recognition and relation extraction tasks. The fine-tuned BERT model can identify entities and their categories in the text and categorize them into multiple triples. A new knowledge graph is then generated based on each triple. The new knowledge graph maintains the same structure as the original knowledge graph but achieves richness and diversity in its content.

2. The method as described in claim 1, characterized in that, The process of generating natural language text from the knowledge graph using Graph Transformer in step 1 is as follows: the triples consisting of subject, relation and object in the original knowledge graph are abstracted into the representation of nodes and edges, and input into Graph Transformer for processing; Graph Transformer identifies the semantic relationship and importance between each node and its neighboring nodes through a multi-head attention mechanism, and updates the representation of the nodes. Then, by normalization and feedforward neural networks, nonlinear transformations are performed to enhance the expressive power and stability of the Graph Transformer model; The updated node representations are input into the decoder, which predicts the next word step by step based on the context, thus obtaining natural language text that conforms to the graph structure.

3. The method as described in claim 2, characterized in that, Step 1, which uses Graph Transformer to generate natural language text from a knowledge graph, specifically involves: first, representing the triples (s, r, o) in the knowledge graph as an initial feature matrix H. (0) Here, s and o represent the subject and object nodes, respectively, while r represents the relationship between them; the initial feature vector of a node is a numerical representation of its attributes or semantic information, which can be generated through One-Hot encoding, embedding vectors, or direct normalization of values; the initial feature vector of an edge represents the relation attributes; the initial feature vectors of nodes and edges are encoded and then organized into an initial feature matrix H. (0) H is a matrix formed by stacking the initial feature vectors of all nodes row-wise, representing the initial state of the entire graph; then this initial feature matrix H is... (0) The input is fed into the Graph Transformer; the Graph Transformer uses a multi-head attention mechanism to compute the semantic relationships between nodes and their neighbors; the method for computing the semantic relationships between nodes and their neighbors is as follows: for each node v i Calculate its attention weight with its neighbor node N(i), using the following formula: Where Q (Query) is the target node v i The query vector; K (Key) is the key vector of neighboring nodes; V (Value) is the value vector of neighboring nodes; d k The dimension is the key vector; the semantic relationship between a node and its neighboring nodes is represented by computed attention weights. Then, the Graph Transformer learns diverse relationships between nodes at each layer and aggregates these relationships into a new node representation, thereby updating the node's feature representation. After updating the node feature representation, the updated feature node representation undergoes nonlinear transformation through layer normalization and a feedforward neural network to enhance the Graph Transformer's ability to express complex relationships. After multiple layers of computation and updates, the final node representation H, containing complete semantic information extracted from the input knowledge graph, is obtained. (L) ; H (L) Input decoder, decoder utilizes H (L) The semantic information contained therein is gradually used to generate natural language text.

4. The method as described in claim 3, characterized in that, The Graph Transformer learns diverse relationships between nodes at each layer and aggregates these relationships into new node representations, thereby updating the node feature representations. After updating the node feature representations, the updated feature node representations undergo nonlinear transformations through layer normalization and a feedforward neural network to enhance the Graph Transformer's ability to express complex relationships. After multiple layers of computation and updates, the final node representation H containing complete semantic information extracted from the input knowledge graph is obtained. (L) The process consists of two stages. composition: a) First, the semantic relationship between the target node and its neighboring nodes is calculated using a multi-head attention mechanism, thereby updating the node's feature representation: H (l) =LayerNorm(H (l-1) +MultiHead(H (l-1) )) The node features H of the previous layer (l-1) As input, the interaction information between the target node and its neighboring nodes is extracted using the MultiHead() multi-head attention mechanism, and residual connections are added to preserve the node's own information; normalization is performed using LayerNorm() to ensure stable feature distribution and accelerate the training process; the final output H (l) Represents the node features of the l-th layer; b) Then, the multilayer perceptron is used to analyze the output features H of the previous stage. (l) Nonlinear mapping and feature enhancement are performed to improve the expressive power of the output features: H (l) '=LayerNorm(H (l) +MLP(H (l) )) The output result H of the first stage (l) The input is fed into a multilayer perceptron (MLP()) to learn higher-dimensional semantic representations, and the original information is preserved through residual connections; finally, LayerNorm() is used for normalization again to further improve the training stability of the model. Each layer l corresponds to one feature update of the network, starting from the initial feature matrix H. (0) Initially, the output feature H of the Lth layer is obtained through layer-by-layer calculation. (L) , representing the final semantic representation of the entire graph; after multiple layers of computation and updates, the final node representation H is obtained. (L) It contains complete semantic information extracted from the input knowledge graph.

5. The method as described in claim 3, characterized in that, In the process of generating natural language text step by step using semantic information, the decoder predicts the probability distribution of generating the next word at each time step based on the current context information: P(w t |w <t ,H (L) )=softmax(W d H (L) ) Among them, W d The weight matrix of the decoder; w t This represents the target word generated at time step t; w <t This represents all words generated before time step t. The decoder employs a stepwise generation strategy, generating the next word at each step by referencing the already generated text and the semantic information stored in the GraphTransformer model. A beam search algorithm is used to optimize the generation path. The beam search retains multiple high-probability candidate paths at each step and selects the optimal path in the final stage.

6. The method as described in claim 5, characterized in that, In the natural language text generation process, the GraphTransformer model is also trained. During the model training phase, supervised learning is performed using text data represented by labeled triples to minimize the cross-entropy loss function of the generated sequence. The parameters of the Graph Transformer model are updated incrementally through backpropagation to improve the semantic accuracy and fluency of the generated text.

7. The method as described in claim 3, characterized in that, In step 2, context-level data augmentation involves adjustments to four main dimensions: sentence length, vocabulary usage, clause structure, and text style—essentially four augmentation strategies—to achieve diverse expansions of text expression. For a given text T, the new text T' generated by context-level data augmentation is represented as T' = f context (T,p), where p represents the enhancement strategy; in sentence length adjustment, the sentence length is expanded or shortened by adding or deleting content to generate different T' to adapt to different contexts and task requirements; in vocabulary usage adjustment, words in the text are replaced; the optimization of clause structure allows for adjustment of the type and position of clauses to generate texts with different structural complexities; in addition, by changing the presentation style of the text, texts adapted to different scenarios can be generated.

8. The method as described in claim 7, characterized in that, After initial context-level augmentation, the augmented text undergoes further processing, specifically using LLM to implement entity replacement to increase text diversity and achieve entity-level data augmentation. Given the entity set E contained in text T', the generated augmented text T″ is represented as T″ = f entity (T',E'), where E' is the set of entities to be replaced; through conditional probability P(e'). i Use `context(T')` to select the appropriate replacement entity `e'`. i The new entity is defined by context(T'), which represents the context of T'. This ensures that the new entity is semantically consistent with the text context. This strategy can generate new texts that contain entities that do not appear in the text T', thereby expanding the diversity of the dataset.

9. The method as described in claim 8, characterized in that, In step 3, during the construction of the new knowledge graph, a pre-trained BERT model is selected as the fine-tuning model to perform entity recognition and relation extraction tasks. For the entity recognition task, the OntoNotes 5.0 dataset was selected for fine-tuning. In this dataset, data is labeled with different entity types, and each training data point is organized as input text and its corresponding label. Assuming the input text is T, its corresponding entity label set is E = {e1, e2, ..., e...} m The goal of fine-tuning is to optimize the parameters of the BERT model so that it can recognize these entities from the input text. The training objective function of the BERT model is expressed as: Among them, y t Let P(y) be the label of the t-th word in a given text T. t |T) is the BERT model's predicted probability for this label; the BERT model parameters are updated using the backpropagation algorithm to minimize this objective function; After fine-tuning the model, the enhanced text data is input into the fine-tuned BERT model for entity recognition. The input to the BERT model is the enhanced text dataset D, and its output is the set of entities E” in the text, with m” of entities and e” for each entity. i Its category C(e') is labeled i The output of entity recognition is represented as: And”={(e'1,C(e'1)),(e'2,C(e'2)),…,(e' m' ,There is' m' ))} For the relation extraction task, the TACRED dataset was selected for fine-tuning. After fine-tuning, context analysis was performed on the identified entities, and the relationships between the entities were determined. Thus, the triples identified by the fine-tuned model are represented as follows: R={(s,r,o)|s∈E”,r∈Relations,o∈E”} Where s is the subject entity, o is the object entity, r is the relationship between them, and Relations represents the set of entity relations. In the process of recognizing triples, the model optimizes the generation of each triple based on the context information in the text, the category of the entity, and the relational knowledge learned during training. Through sequence labeling and classification methods, the model can effectively extract triple data in the form of (s,r,o). Once all triples are generated, the triples are divided into multiple sets based on the categories of the entities within them. Each set is used to generate a corresponding new knowledge graph G' = (V', E'), where V' represents the set of nodes in the new knowledge graph, and E' represents the set of edges. The new knowledge graph is represented as follows: G'=(V',E')=({s1,s2,…,s n },{(s i ,r j ,o k )|(s i ,r j ,o k )∈R}) Ultimately, the generated new graph maintains structural consistency with the original knowledge graph, while adding diversity and richness in content, and conforms to reality.

Images