Knowledge graph interpretive text generation method based on graph neural network hierarchical structure, intelligent system and program product

Abstract

The invention discloses a knowledge graph interpretive text generation method, system and program based on a graph neural network hierarchical structure. The method comprises the following steps: inputting a graph and corresponding node features, edge features and an adjacent matrix thereof, respectively converting the node features and the edge features into vectors, fusing the node features and the edge features represented by the vectors, and encoding the fused vectors; dividing an input graph into a plurality of sub-graphs, and performing pooling operation on the divided sub-graphs and the input graph respectively; respectively decoding the coded fusion vector, the feature vector corresponding to each sub-graph and the full-graph feature vector to obtain a node text, a sub-graph text and a full-graph text; and sequentially generating the node text, each sub-image text and the full-image text into an atlas interpretation text with a layered structure. According to the method, the knowledge graph descriptive text with high organization and coherence can be generated, and the generated text can accurately reflect the structure and semantics of the graph from details to summary and from local to global.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, intelligent system and program product for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure. Background Art

[0002] With the continuous expansion of graph data applications in various fields, graph neural networks have become the core tool for analyzing and processing graph data. Graph data is a typical non-Euclidean data structure consisting of nodes and edges. It has been widely used in many fields such as knowledge graphs, social networks, molecular graphs, and recommendation systems. Traditional machine learning methods, such as convolutional neural networks and recurrent neural networks, have certain limitations when processing graph data because these methods usually assume that the data is regular (such as the grid structure of image or text data), while the structure of graph data is more flexible and complex.

[0003] As graph neural network technology matures, the representation learning capabilities of graphs have gradually expanded and improved, but how to combine graph learning with natural language generation technology, especially expressing the structure and semantics of graphs through text descriptions generated by graphs, is still a forward-looking technical research direction.

[0004] In recent years, text generation models based on deep learning have made significant progress, especially the technology of generating text based on recurrent neural networks and long short-term memory networks. Since natural language generation is one of the important research directions of natural language processing, it aims to generate natural language description text from structured data. Currently, people convert the input feature vector into natural language text, making the long short-term memory network model widely used in tasks such as dialogue systems, text summarization and image description generation.

[0005] Through summary, this patent found that the main existing shortcomings in graph structure text generation technology include:

[0006] Graph data usually has a complex hierarchical structure, including nodes, edges, subgraphs, and the entire graph. When generating descriptive text for a graph, it is still a challenge to accurately convert this hierarchical information into structured text. Existing text generation methods often lack the ability to handle this hierarchical structure, resulting in the generated explanatory text often being only a simplified description of a single node or edge, lacking a descriptive language for the entire graph, that is, lacking a hierarchical graph structure expression.

[0007] In the processing and analysis of graph data, interpretability is a crucial factor. Especially in some high-risk fields, such as medicine, finance, and education, the generated text of the graph must be able to provide detailed explanations to help users understand the generation process and decision results of the model. When generating graph explanatory text, existing technologies often do not fully consider the transparency and interpretability of the generation process.

[0008] In short, the relationships between nodes and edges in the science and technology graph structure are complex and changeable. How to accurately and clearly express these complex relationships in textual form to ensure that the generated text has high coherence and readability, as well as transparency and explainability, is still a technical problem that needs to be solved urgently. Summary of the invention

[0009] The present invention provides a method, intelligent system and program for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure, so as to solve the difficult problems existing in the above-mentioned prior art.

[0010] To achieve the above object, the present invention provides a method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure, comprising the following steps:

[0011] Input a graph and its corresponding node features, edge features and adjacency matrix, convert the node features and edge features into vectors respectively, fuse the node features and edge features represented by the vectors to obtain a fused vector having node information and corresponding edge information, and encode the fused vector;

[0012] The input graph is divided into multiple sub-graphs, and pooling operations are performed on the divided sub-graphs and the input graph respectively to obtain the feature vector corresponding to each sub-graph and the feature vector of the whole graph;

[0013] Decode the encoded fusion vector, the feature vector corresponding to each subgraph, and the full-graph feature vector respectively to obtain the node text corresponding to the fusion vector, the subgraph text corresponding to each subgraph feature vector, and the full-graph text corresponding to the full-graph feature vector;

[0014] The node text, each subgraph text, and the whole graph text are sequentially generated into a total graph explanation text to form a graph explanation text with a hierarchical structure.

[0015] In some embodiments, encoding the fusion vector comprises the following steps:

[0016] According to the adjacency matrix, the features of each node of the fusion vector and the features of its neighboring nodes are weighted summed multiple times, and the results of each weighted summation are spliced, and the spliced ​​linear transformation is mapped to the target dimensional space.

[0017] In some embodiments, after performing weighted summation on the features of each node of the fusion vector and the features of its neighboring nodes each time, the result of the weighted summation is further transformed through a feedforward network.

[0018] In some embodiments, the feedforward network includes two linear transformation layers and one activation function layer; the result after each weighted summation is sequentially processed by the first layer linear transformation, the activation function layer, and the second layer linear transformation before output.

[0019] In some embodiments, before the result after each weighted summation is transformed through the feedforward network, the result after each weighted summation is subjected to residual connection and normalization processing.

[0020] In some embodiments, the residual connection processing includes the following steps: adding the fusion vector to the result of weighted summation.

[0021] In some embodiments, a multi-head self-attention mechanism is used to perform multiple weighted summations on the features of each node of the fusion vector and the features of its neighboring nodes.

[0022] In some embodiments, the fused vector is normalized before being encoded.

[0023] According to another aspect of the present invention, an intelligent system is provided for implementing the above-mentioned method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure, including an embedding layer, a fusion layer, a self-attention layer, a subgraph partitioning and pooling layer, a full-graph pooling layer, and a long short-term memory network layer.

[0024] The embedding layer converts the node features and edge features corresponding to the input graph into vector representations in a continuous space respectively; the fusion layer fuses the node features and edge features represented by the vector output by the embedding layer, and performs normalization after fusion; the self-attention layer uses a multi-head self-attention mechanism to perform multiple weighted summations on the features of each node of the fusion vector output by the fusion layer and the features of its neighboring nodes, performs residual connection and normalization on the results after each weighted summation, further transforms the results after each residual connection and normalization through a feedforward network, and splices the outputs of each feedforward network. The subsequent linear transformation is mapped to the target dimensional space; the subgraph division and pooling layer divides the input graph into multiple subgraphs, performs pooling operations on the divided subgraphs, and obtains a feature vector corresponding to each subgraph; the full-graph pooling layer performs pooling operations on the input graph to obtain a full-graph feature vector; the long short-term memory network layer respectively decodes the fusion vector output by the self-attention layer, the feature vector corresponding to the subgraph output by the subgraph division and pooling layer, and the full-graph feature vector output by the full-graph pooling layer, and generates a total graph explanation text from the decoded node text, subgraph text, and full-graph text in sequence, forming a graph explanation text with a hierarchical structure.

[0025] According to another aspect of the present invention, a program product is also provided, including a computer program / instruction, which, when executed by a processor, implements the steps of a method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure as described above.

[0026] Implementing one of the above technical solutions of the present invention has the following advantages or beneficial effects:

[0027] The present invention can generate highly organized and coherent graph description text by accurately modeling the hierarchical structure of the graph, accurately conveying the structural information of the graph. The generated text has clear hierarchy from nodes to subgraphs to the entire graph, showing the complexity and details of the graph. It not only has high coherence, but also can explain in detail the important role of each node and subgraph, enhancing the transparency and interpretability of the model. Further beneficial effects are as follows:

[0028] 1. Efficient fusion of graph structure and node information: The present invention can efficiently fuse node features, edge features and adjacency relationships in the graph in the encoder part through technical means such as embedding layer, attention mechanism and pooling operation. Through the multi-head self-attention mechanism, the model can flexibly capture the complex dependencies between nodes and effectively integrate the information of nodes and their neighboring nodes, thereby providing richer and more accurate feature representation for subsequent explanatory text generation.

[0029] 2. Accurate graph structure representation: It can accurately capture the features of nodes and edges in the graph, and generate graph feature representations containing more comprehensive information through multi-layer processing.

[0030] 3. Enhanced capture of relationships between nodes: The self-attention mechanism enables the model to dynamically weight nodes based on their relative importance to neighboring nodes, thereby improving the ability to express complex relationships between nodes.

[0031] 4. Hierarchical graph description generation: The present invention can accurately generate hierarchical text descriptions through a multi-level generation process from nodes to subgraphs and then to the full graph. First, a local description of the node is generated, then a description of the subgraph is generated, and finally a description of the full graph is generated, forming a clearly structured and gradually unfolding text sequence. Such a hierarchical description can not only express the personalized information of the node, but also show the global pattern and complex structure of the entire graph.

[0032] 5. Progressive generation from local to global: The text generation process is carried out layer by layer with clear levels, avoiding the problem of generating scattered and incoherent text in the existing technology.

[0033] 6. Expressing the complexity of the graph: By describing subgraphs and the entire graph, the model can accurately display the global structure and macro relationships of the graph, improving the richness and accuracy of the graph description. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A flowchart of a method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure according to an embodiment of the present invention;

[0036] Figure 2 The figure is a network structure diagram of an intelligent system according to an embodiment of the present invention.

[0037] In the figure: 501, embedding layer; 502, fusion layer; 503, self-attention layer; 504, sub-graph division and pooling layer; 505, full-graph pooling layer; 506, long short-term memory network layer. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] Embodiment 1:

[0040] Figure 1 The figure shows a method for generating explanatory text from a knowledge graph based on a hierarchical structure of a graph neural network. The method for generating explanatory text from a graph includes the following steps:

[0041] S100, input a graph and its corresponding node features, edge features and adjacency matrix, convert the node features and edge features into vector representations respectively, fuse the node features and edge features represented by the vectors to obtain a fused vector having node information and corresponding edge information, and encode the fused vector.

[0042] The input node features can be converted into vector representations in continuous space through an embedding layer (such as the embedding layer corresponding to the neural network in the figure). This layer embeds the features of each node into a hidden dimension space to form an embedded representation (vector representation) of the node. The features of the two edges connecting the nodes are embedded to form an embedded representation of the edge. The edge embedding is then added to the node embedding to obtain a fused embedding that contains the information of the node and the information of the connected edges.

[0043] Encoding the fusion vector includes the following steps: according to the adjacency matrix, weighted summing the features of each node of the fusion vector with the features of its neighboring nodes for multiple times, concatenating the results of each weighted summation, and linearly transforming and mapping them to the target dimensional space after concatenation. Among them, a multi-head self-attention mechanism is used to weighted sum the features of each node of the fusion vector with the features of its neighboring nodes for multiple times.

[0044] It should be noted that the multi-head self-attention mechanism uses N layers. In each layer, this method applies the multi-head self-attention mechanism to calculate the relationship between nodes. The features of each node are weighted summed with the features of its neighboring nodes, so that the node features can be updated according to the relative relationship between the nodes. In order to efficiently parallelize the process, the multi-head self-attention mechanism is introduced, that is, the attention is calculated in parallel in different subspaces, and then the results are merged.

[0045] The encoding process uses a multi-head self-attention mechanism to calculate the relationship between nodes. The features of each node are weighted and summed with the features of its neighboring nodes, so as to update the representation of the node according to the relationship between the nodes. In order to improve the computational efficiency, a multi-head mechanism is adopted, that is, multiple attention heads are calculated in parallel, each head focuses on a different subspace, and finally the results are merged. Each head independently calculates the query, key and value, and obtains the updated representation of the node through weighted summation.

[0046] After multiple layers of self-attention mechanism processing, the final encoding result of each node is obtained. This encoding combines the information of each node itself and its relationship with other nodes.

[0047] After the multi-head self-attention calculation, residual connections are added and then normalized. Residual connections add the input and output to help stabilize the training process. Normalization further adjusts the distribution of the output, thereby facilitating model training. The self-attention output of each layer is passed into a feedforward neural network for further transformation. This neural network usually includes two linear transformations and an activation function. Through the feedforward network, the node representation is further updated, thereby improving the model's expressiveness.

[0048] S200, dividing the input graph into multiple sub-graphs, performing pooling operations on the divided sub-graphs and the input graph respectively, and obtaining a feature vector corresponding to each sub-graph and a feature vector of the entire graph.

[0049] In a specific example, for step S200, the whole image is divided into k sub-images by using the K-means algorithm, and an average pooling operation is performed on each sub-image to obtain a feature vector of each sub-image. The pooled sub-image feature vector represents the information of the entire sub-image.

[0050] It should be noted that the entire graph is divided into multiple subgraphs using the K-means algorithm. For each subgraph, a pooling operation is used to extract its features. K-means clustering divides the nodes in the graph into several clusters, each cluster represents a subgraph. Then, the features of the subgraph are obtained by calculating the average value of the node representation in each subgraph.

[0051] Furthermore, the node features of the entire graph are pooled to obtain the global feature representation of the graph. The full-graph pooling operation aggregates the feature vectors of all nodes in the graph to obtain the global feature vector of the entire graph. The full-graph pooling operation uses average pooling:

[0052] S300, respectively decode the encoded node vector, the feature vector corresponding to each subgraph, and the full-graph feature vector to obtain the node text corresponding to the fusion vector, the subgraph text corresponding to each subgraph feature vector, and the full-graph text corresponding to the full-graph feature vector;

[0053] In a specific example, for the node text generation in step S300, the node feature matrix output by the encoding is accepted, and a text description is generated for each node. The generated text generally includes the attributes, category of the node and its relationship with neighboring nodes. The feature vector is converted into text through the LSTM layer.

[0054] Furthermore, for the full-graph text generation in step S300, the subgraph features reflect the local hierarchical structure of the graph, and the decoder generates a local description of the graph based on the subgraph features, and converts the feature vector into text through the LSTM layer.

[0055] Furthermore, for the full-image text generation in step S300, the global features reflect the structure of the entire image, and the decoder generates a global description of the image based on these features. The feature vector is converted into text through the LSTM layer. The input of the full-image text generation is the embedded representation of the full image, and the global description of the image is generated.

[0056] The features of nodes, subgraphs, and full graphs are converted into text through the LSTM model. Finally, the generated text passes through a Softmax layer, is mapped into the vocabulary, and its text description is output.

[0057] S400, generating a total graph explanation text from the node text, each subgraph text, and the whole graph text in sequence, to form a graph explanation text with a hierarchical structure.

[0058] In a specific example, for the generation of the overall graph explanation text of step S400, the order of the generated text mainly depends on the structure from local to global, from details to summary, to ensure the hierarchy and coherence of the information. Specifically, the following order can be adopted: first describe the details of a single node, including the node's attributes, categories, and relationships with adjacent nodes. This part usually contains very specific information. Next, describe the subgraphs in the graph, which may represent specific modules, groups, or features, and have certain internal structures and relationships. Finally, generate an overall description of the entire graph, emphasizing the pattern, structure, and overall relationship of the entire graph.

[0059] Based on the above embodiment, the method is also included in the training process, and the goal is to minimize the difference between the generated text and the real text. The cross entropy loss is used to measure the similarity between the generated text and the real text. The loss function calculates the text generation loss of each node, subgraph and full graph respectively, and then sums these losses according to the weighted coefficient to obtain the total loss. The Adam optimizer is used for training. The Adam optimizer can combine momentum and adaptive learning rate, and is suitable for training deep neural networks. During the training process, the optimized parameters include the embedding matrix of nodes and edges, the weight of the attention mechanism, the weight of the feedforward network, the parameters of the LSTM model, etc.

[0060] The method of this embodiment can generate highly organized and coherent graph description text by accurately modeling the hierarchical structure of the graph, and accurately convey the structural information of the graph. The generated text has a clear hierarchy from nodes to subgraphs to the entire graph, showing the complexity and details of the graph. It not only has a high degree of coherence, but also can explain in detail the role of each node and subgraph, enhancing the transparency and interpretability of the model.

[0061] In summary, the technical solution of combining graph neural network and text generation proposed by the above method of this embodiment, by inputting node features and adjacency matrix, using embedding layer to transform node features, and then using multi-head self-attention mechanism and feedforward neural network to encode nodes. Finally, the embedding representation of each node and the global feature representation of the graph are obtained. The K-means algorithm is used to divide the whole graph into multiple subgraphs, and each subgraph is pooled to extract subgraph features. Finally, the node features, subgraph features and full-graph features are spliced ​​as the input of the decoder. The decoder uses the LSTM layer to generate text descriptions of nodes, subgraphs and full graphs, and generates a structured description of the graph from local to global by layer-by-layer generation. The generated text can accurately reflect the structure and semantics of the graph from details to summary, from local to global.

[0062] Embodiment 2

[0063] like Figure 2 As shown, this embodiment provides an intelligent system for implementing the knowledge graph explanatory text generation method based on the graph neural network hierarchical structure described in Embodiment 1, including an embedding layer 501, a fusion layer 502, a self-attention layer 503, a subgraph partitioning and pooling layer 504, a full-graph pooling layer 505, and a long short-term memory network layer 506. Among them:

[0064] The embedding layer 501 converts the node features and edge features corresponding to the input graph into vector representations in a continuous space.

[0065] The fusion layer 502 fuses the node features and edge features represented by the vectors output by the embedding layer 501, and performs normalization after fusion.

[0066] The self-attention layer 503 uses a multi-head self-attention mechanism to perform multiple weighted summations on the features of each node of the fusion vector output by the fusion layer 502 and the features of its neighboring nodes, and performs residual connection and normalization processing on the result after each weighted summation. The result after each residual connection and normalization processing is further transformed through a feedforward network, and the output of each feedforward network is spliced, and the linear transformation after splicing is mapped to the target dimensional space.

[0067] The subgraph partitioning and pooling layer 504 partitions the input graph into multiple subgraphs, performs pooling operations on the partitioned subgraphs, and obtains a feature vector corresponding to each subgraph. The K-means algorithm is used to partition the entire graph into k subgraphs.

[0068] The full-image pooling layer 505 performs a pooling operation on the input image to obtain a full-image feature vector.

[0069] The long short-term memory network layer 506 decodes the fusion vector output by the self-attention layer 503, the feature vector corresponding to the subgraph output by the subgraph partitioning and pooling layer 504, and the full-graph feature vector output by the full-graph pooling layer 505, and sequentially generates a total graph explanation text from the decoded node text, subgraph text, and full-graph text, thereby forming a graph explanation text with a hierarchical structure.

[0070] Based on the above embodiment, the embedding layer 501 may further include a node embedding layer and an edge embedding layer. The node embedding layer is used to convert the node features corresponding to the input graph into a vector representation in a continuous space, and the edge embedding layer converts the edge features into a vector representation in a continuous space.

[0071] The feedforward network includes two linear transformation layers and one activation function layer; the result after each weighted summation is sequentially processed by the first layer of linear transformation, the activation function layer, and the second layer of linear transformation before output.

[0072] The computing models of the above-mentioned embedding layer 501, fusion layer 502, self-attention layer 503, sub-graph partitioning and pooling layer 504, full-graph pooling layer 505, and long short-term memory network layer 506 and their related settings, as well as the training and optimization of the intelligent system model correspond to the first embodiment and will not be repeated here.

[0073] Embodiment 3

[0074] The embodiment also provides a program product, including a computer program / instruction, which, when executed by a processor, implements the steps of a method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure as described in the first embodiment above. Since the embodiment of the computer program product part corresponds to the embodiment of the method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure, please refer to the description of the first embodiment above for the embodiment of the computer program product part, and will not be repeated here.

[0075] Those skilled in the art can understand that the accompanying drawings are only schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.

[0076] Those skilled in the art can understand that the modules in the device in the embodiment can be distributed in the device in the embodiment according to the description of the embodiment, or can be changed accordingly and located in one or more devices different from the embodiment. The modules in the above embodiment can be combined into one module, or can be further divided into multiple sub-modules.

[0077] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure, characterized in that: The following steps are involved: Input a graph and its corresponding node features, edge features and adjacency matrix, convert the node features and edge features into vector representations respectively, fuse the node features and edge features represented by the vectors to obtain a fused vector having node information and corresponding edge information, and encode the fused vector; The input graph is divided into multiple sub-graphs, and pooling operations are performed on the divided sub-graphs and the input graph respectively to obtain the feature vector corresponding to each sub-graph and the feature vector of the whole graph; The encoded fusion vector, the feature vector corresponding to each subgraph, and the full-graph feature vector are decoded respectively to obtain the node text corresponding to the fusion vector, the subgraph text corresponding to each subgraph feature vector, and the full-graph text corresponding to the full-graph feature vector; The node text, each subgraph text, and the whole graph text are sequentially generated into a total graph explanation text to form a graph explanation text with a hierarchical structure.

2. According to claim 1, a method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure is characterized in that: Encoding the fusion vector comprises the following steps: According to the adjacency matrix, the features of each node of the fusion vector and the features of its neighboring nodes are weighted summed multiple times, and the results of each weighted summation are spliced, and the spliced ​​linear transformation is mapped to the target dimensional space.

3. According to claim 2, a method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure is characterized in that: After performing weighted summation on the features of each node of the fusion vector and the features of its neighboring nodes each time, the result of the weighted summation is further transformed through a feedforward network.

4. According to claim 3, a method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure is characterized in that: The feedforward network includes two linear transformation layers and an activation function layer; the result after each weighted summation is sequentially processed by the first layer linear transformation, the activation function layer, and the second layer linear transformation before output.

5. According to claim 3, a method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure is characterized in that: Before the result after each weighted summation is transformed through the feedforward network, the result after each weighted summation is subjected to residual connection and normalization processing.

6. A method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure according to any one of claim 5, characterized in that: The residual connection processing comprises the following steps: The fusion vector is added to the result of the weighted summation.

7. A method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure according to any one of claim 2, characterized in that: A multi-head self-attention mechanism is used to perform multiple weighted summations on the features of each node of the fusion vector and the features of its neighboring nodes.

8. A method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure according to any one of claims 1 to 7, characterized in that: Before encoding the fused vector, it is normalized.

9. An intelligent system, characterized in that: A method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure according to any one of claims 1 to 8, comprising an embedding layer, a fusion layer, a self-attention layer, a subgraph partitioning and pooling layer, a full-graph pooling layer, and a long short-term memory network layer; The embedding layer converts the node features and edge features corresponding to the input graph into vector representations in a continuous space respectively; The fusion layer fuses the node features and edge features represented by the vectors output by the embedding layer, and performs normalization processing after fusion; The self-attention layer uses a multi-head self-attention mechanism to perform multiple weighted summations on the features of each node of the fusion vector output by the fusion layer and the features of its neighboring nodes, performs residual connection and normalization processing on the result after each weighted summation, further transforms the result after each residual connection and normalization processing through a feedforward network, splices the output of each feedforward network, and linearly transforms and maps it to the target dimensional space after splicing; The subgraph partitioning and pooling layer partitions the input graph into multiple subgraphs, performs pooling operations on the partitioned subgraphs, and obtains a feature vector corresponding to each subgraph; The full-image pooling layer performs a pooling operation on the input image to obtain a full-image feature vector; The long short-term memory network layer respectively decodes the fusion vector output by the self-attention layer, the feature vector corresponding to the subgraph output by the subgraph division and pooling layer, and the full-graph feature vector output by the full-graph pooling layer, and sequentially generates a total graph explanation text from the decoded node text, subgraph text, and full-graph text to form a graph explanation text with a hierarchical structure.

10. A program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of a method for generating explanatory text from a knowledge graph based on a graph neural network hierarchical structure as described in any one of claims 1 to 8 are implemented.

Images