Information reasoning method and device and computer equipment

Abstract

The invention relates to the field of natural language processing, in particular to an information reasoning method and device, computer equipment and a storage medium, multi-scale transformation is carried out on a knowledge graph, a multi-scale zoom graph of the knowledge graph is constructed, multi-scale interaction is carried out by combining a to-be-reasoned sub-graph and a multi-scale transformation weight matrix constructed based on the to-be-reasoned sub-graph, and the multi-scale transformation weight matrix of the to-be-reasoned sub-graph is obtained. According to the method, adjacent node information from different scales is aggregated, and the information of different scales is fully utilized to construct a more accurate retrieval sub-graph for graph information reasoning, so that the accuracy of accurate information reasoning on the graph is improved.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing, and in particular to an information reasoning method, apparatus, computer equipment and storage medium. Background Art

[0002] Large Language Models (LLMs) have shown excellent performance in understanding and summarizing unstructured text. Given the powerful ability of graphs in representing structured data, it is natural to combine LLMs with graphs to achieve better performance in the field of natural language processing, including classification and generation tasks.

[0003] Recently, there has been a surge in research on methods for effectively utilizing graph structural information to enhance the capabilities of LLMs. These methods enhance the structural information of LLMs by accurately matching the obtained subgraphs. Some of these studies combine graph embedding with LLMs. GNP uses soft hints to provide structural information modeled by graph neural networks (GNNs) to LLMs, while GraphAdapter combines GNNs as adapters with LLMs. Other studies enable LLMs to reason based on graph structures and integrate structural information into the reasoning process. ToG and MD-QA directly query graphs through LLMs. However, the exact matching of the target graph only utilizes local information in the large graph, ignoring the global relevant information in the entire graph, which reduces the accuracy of accurate information reasoning about the graph. Summary of the invention

[0004] Based on this, the purpose of the present invention is to provide an information reasoning method, apparatus, computer equipment and storage medium to perform multi-scale transformation on the knowledge graph, construct a multi-scale zoom graph of the knowledge graph, and perform multi-scale interaction in combination with the sub-graph to be reasoned and the multi-scale transformation weight matrix constructed based on the sub-graph to be reasoned, aggregate neighboring node information from different scales, make full use of information at different scales, and construct a more accurate retrieval sub-graph for graph information reasoning, thereby improving the accuracy of accurate information reasoning on the graph.

[0005] In a first aspect, an embodiment of the present application provides an information reasoning method, the method comprising the following steps:

[0006] Obtaining a knowledge graph, a subgraph to be inferred, and a graph-enhanced language model, wherein the graph-enhanced language model includes a multi-scale transformation module, a multi-scale interaction module, and a reasoning module; the multi-scale interaction module includes a matrix construction unit, an interaction unit, and a graph core retrieval unit;

[0007] Inputting the knowledge graph into the multi-scale transformation module for transformation processing and feature extraction to obtain scaling graphs of several scales and feature matrices of the scaling graphs;

[0008] Inputting the subgraph to be inferred into the matrix construction unit to construct a transformation weight matrix to obtain transformation weight matrices of several scales;

[0009] Obtaining a feature matrix of the subgraph to be inferred, a retrieval subgraph set of a previous scale, and feature matrices of several retrieval subgraphs in the retrieval subgraph set; inputting the retrieval subgraph set of a previous scale, feature matrices of several retrieval subgraphs in the retrieval subgraph set, the subgraph to be inferred, the feature matrix of the subgraph to be inferred, and a transformation weight matrix of the current scale into the interactive unit for interactive processing, and obtaining an interactive feature matrix of the subgraph to be inferred at the current scale;

[0010] Inputting the zoomed graph of the current scale and the feature matrix of the zoomed graph, the subgraph to be inferred and the interactive feature matrix of the subgraph to be inferred into the graph core retrieval unit to construct a retrieval subgraph, obtaining a retrieval subgraph set of the current scale, and repeating the process until a retrieval subgraph set of the last scale is obtained, thereby obtaining retrieval subgraph sets of several scales;

[0011] The subgraph to be inferred and a set of retrieval subgraphs of several scales are input into the reasoning module for information reasoning to obtain the information reasoning result of the subgraph to be inferred.

[0012] In a second aspect, an embodiment of the present application provides an information reasoning device, including:

[0013] A data acquisition module, used to obtain a knowledge graph, a subgraph to be inferred, and a graph-enhanced language model, wherein the graph-enhanced language model includes a multi-scale transformation module, a multi-scale interaction module, and a reasoning module; the multi-scale interaction module includes a matrix construction unit, an interaction unit, and a graph core retrieval unit;

[0014] A multi-scale feature extraction module, used to input the knowledge graph into the multi-scale transformation module for transformation processing and feature extraction, and obtain scaling graphs of several scales and feature matrices of the scaling graphs;

[0015] A transformation weight matrix construction module, used for inputting the subgraph to be inferred into the matrix construction unit to construct a transformation weight matrix, and obtaining transformation weight matrices of several scales;

[0016] A multi-scale interactive processing module is used to obtain the feature matrix of the subgraph to be inferred, the retrieval subgraph set of the previous scale, and the feature matrices of several retrieval subgraphs in the retrieval subgraph set; the retrieval subgraph set of the previous scale, the feature matrices of several retrieval subgraphs in the retrieval subgraph set, the subgraph to be inferred, the feature matrix of the subgraph to be inferred, and the transformation weight matrix of the current scale are input into the interactive unit for interactive processing to obtain the interactive feature matrix of the subgraph to be inferred at the current scale;

[0017] A retrieval subgraph construction module is used to input the zoomed graph of the current scale and the feature matrix of the zoomed graph, the subgraph to be inferred and the interactive feature matrix of the subgraph to be inferred into the graph core retrieval unit to construct a retrieval subgraph, obtain a retrieval subgraph set of the current scale, and repeat the process until a retrieval subgraph set of the last scale is obtained, thereby obtaining retrieval subgraph sets of several scales;

[0018] The subgraph information reasoning module is used to input the subgraph to be reasoned and a set of search subgraphs of several scales into the reasoning module for information reasoning to obtain the information reasoning result of the subgraph to be reasoned.

[0019] In a third aspect, an embodiment of the present application provides a computer device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the information reasoning method described in the first aspect are implemented.

[0020] In a fourth aspect, an embodiment of the present application provides a storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the information reasoning method described in the first aspect are implemented.

[0021] In an embodiment of the present application, an information reasoning method, apparatus, computer device, and storage medium are provided to perform multi-scale transformation on a knowledge graph, construct a multi-scale zoom graph of the knowledge graph, and perform multi-scale interaction in combination with a subgraph to be reasoned and a multi-scale transformation weight matrix constructed based on the subgraph to be reasoned, aggregate neighboring node information from different scales, make full use of information at different scales, and construct a more accurate retrieval subgraph for graph information reasoning, thereby improving the accuracy of accurate information reasoning on the graph.

[0022] For better understanding and implementation, the present invention is described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 A flowchart of an information reasoning method provided for one embodiment of the present application;

[0024] Figure 2 A schematic diagram of the process of S2 in the information reasoning method provided in one embodiment of the present application;

[0025] Figure 3 A schematic diagram of the process of S3 in the information reasoning method provided in one embodiment of the present application;

[0026] Figure 4 A schematic diagram of the process of S4 in the information reasoning method provided in one embodiment of the present application;

[0027] Figure 5 A schematic diagram of the process of S4 in the information reasoning method provided in one embodiment of the present application;

[0028] Figure 6 A flowchart of an information reasoning method provided for another embodiment of the present application;

[0029] Figure 7 A schematic diagram of the structure of an information reasoning device provided in one embodiment of the present application;

[0030] Figure 8 A schematic diagram of the structure of a computer device provided for one embodiment of the present application. DETAILED DESCRIPTION

[0031] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0032] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0033] It should be understood that, although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" / "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0034] See also Figure 1 , Figure 1 A flowchart of an information reasoning method provided in one embodiment of the present application, the method comprising the following steps:

[0035] S1: Obtain the knowledge graph, the subgraph to be inferred, and the graph-enhanced language model.

[0036] The executor of the information reasoning method is the reasoning device of the information reasoning method (hereinafter referred to as the reasoning device). The reasoning device can be implemented by software and / or hardware, and the information reasoning method can be implemented by software and / or hardware. The reasoning device can be composed of two or more physical entities, or it can be composed of one physical entity. The hardware pointed to by the reasoning device essentially refers to computer equipment. For example, the reasoning device can be a computer, a mobile phone, a tablet or an interactive tablet. In an optional embodiment, the reasoning device can specifically be a server, or a server cluster composed of multiple computer devices.

[0037] In this embodiment, the reasoning device can obtain the knowledge graph and the subgraph to be reasoned input by the user, and can also obtain the knowledge graph, the subgraph to be reasoned and the graph enhanced language model from a preset database, wherein the graph enhanced language model includes a multi-scale transformation module, a multi-scale interaction module and a reasoning module; the multi-scale interaction module includes a matrix construction unit, an interaction unit and a graph core retrieval unit.

[0038] S2: Input the knowledge graph into the multi-scale transformation module for transformation processing and feature extraction to obtain scaling graphs of several scales and feature matrices of the scaling graphs.

[0039] In this embodiment, the inference device inputs the knowledge graph into the multi-scale transformation module for transformation processing and feature extraction, and obtains scaled graphs of several scales and feature matrices of the scaled graphs to capture the multi-scale information of the knowledge graph. The larger scale maintains the global information of the graph, while the smaller scale retains the details of the fine-grained substructure.

[0040] See also Figure 2 , Figure 2 The flowchart of S2 in the information reasoning method provided in one embodiment of the present application includes steps S21 to S23, which are as follows:

[0041] S21: Aggregate and classify the nodes in the knowledge graph according to the preset scaling factors of several scales, convert the knowledge graph into scaling graphs of several scales, and obtain scaling graphs of several scales.

[0042] In this embodiment, the inference device aggregates and classifies the nodes in the knowledge graph according to the preset scaling factors of several scales, converts the knowledge graph into a scaling graph of several scales, and obtains the scaling graph of several scales, wherein the scaling graph includes several nodes, and the nodes include important nodes and non-important nodes.

[0043] Specifically, the inference device calculates the value of the scaling factor r according to k, aggregate the nodes in the knowledge graph, convert the knowledge graph into a scaled graph of several scales, and obtain the scaled graph of several scales. To ensure the retention of key information in the graph, the inference device scores the nodes in the scaled graph through an importance function, which uses task-specific indicators (such as information entropy, degree or other related indicators) to evaluate the nodes. According to the scaling factor r k Keep the top k nodes with the highest scores, denoted as represents the set of important nodes retained after scaling, while the remaining unimportant nodes are represented as

[0044] S22: converting text attributes of a plurality of nodes in the zoomed graphs of a plurality of scales, and constructing important node feature matrices and non-important node feature matrices of the zoomed graphs of a plurality of scales.

[0045] In this embodiment, the inference device uses a pre-trained language model (PLM) to convert text attributes of several nodes in zoom graphs of several scales, and constructs important node feature matrices and non-important node feature matrices of the zoom graphs of several scales to promote alignment and interaction between multi-scale graphs.

[0046] S23: Obtain node matching relationship matrices of zoom graphs of several scales, and obtain feature aggregation matrices of zoom graphs of several scales according to the node matching relationship matrices of zoom graphs of several scales, important node feature matrices, non-important node feature matrices, and a preset feature aggregation algorithm as feature matrices of the zoom graphs of several scales.

[0047] In order to reduce the loss of graph information caused by scale transformation, in this embodiment, the inference device obtains the node matching relationship matrix of the scaled graphs of several scales, and the node matching relationship matrix includes the node matching relationship vectors between several nodes, and the node matching relationship vector is used to indicate whether there is a matching relationship between the nodes. The inference device obtains the feature aggregation matrix of the scaled graphs of several scales according to the node matching relationship matrix of the scaled graphs of several scales, the important node feature matrix, the non-important node feature matrix and the preset feature aggregation algorithm, as the feature matrix of the scaled graph, wherein the feature aggregation algorithm is:

[0048]

[0049] Where M k is the node matching relationship matrix of the kth scale, is the node matching relationship vector between the ith node and the jth node in the node matching relationship matrix of the kth scale, is the node matching relationship matrix M k The degree vector of the i-th node in kM is the node matching relationship matrix k The degree matrix of is the feature matrix of the non-important nodes of the scaled graph at the kth scale, X ′ k Based on matching relationship The cumulative feature matrix of is the important node feature matrix of the k-th scaled graph, is the similarity feature matrix between the feature matrix of important nodes and the feature matrix of unimportant nodes in the scaled graph of the kth scale, I is the unit matrix, is the feature aggregation matrix of the k-th scaled graph.

[0050] S3: Inputting the subgraph to be inferred into the matrix construction unit to construct a transformation weight matrix to obtain transformation weight matrices of several scales.

[0051] In order to better utilize the graph information in the subgraph to be inferred and promote the information flow between the multi-scale zoom graph and the subgraph to be inferred, in this embodiment, the inference device inputs the subgraph to be inferred into the matrix construction unit to construct a transformation weight matrix to obtain transformation weight matrices of several scales.

[0052] See also Figure 3 , Figure 3 The flowchart of S3 in the information reasoning method provided in one embodiment of the present application includes steps S31 to S32, which are specifically as follows:

[0053] S31: extracting the text supplementary information of the subgraph to be inferred, performing multi-scale embedding processing on the text supplementary information, and obtaining representations of the text supplementary information at several scales.

[0054] In this embodiment, the inference device uses a pre-trained language model (PLM) to extract the text supplementary information of the subgraph to be inferred, performs multi-scale embedding processing on the text supplementary information, and obtains text supplementary information representations at several scales.

[0055] S32: Obtain transformation weight matrices of several scales according to the text supplementary information representations of several scales and a preset transformation weight matrix calculation algorithm.

[0056] In this embodiment, the inference device adopts a graph neural network (GNN) to integrate the text supplementary information representations of several scales into the information aggregation process of the graph neural network. According to the text supplementary information representations of several scales and a preset transformation weight matrix calculation algorithm, combined with the text supplementary information representation, the initial weight matrix is ​​modulated by scaling and translating to capture the common global structure of the graph, and a specific in-scale transformation weight matrix is ​​constructed to obtain transformation weight matrices of several scales, wherein the transformation weight matrix calculation algorithm is:

[0057]

[0058] In the formula, is the transformation weight matrix of the kth scale, W intra is the initial weight matrix, ⊙ is the bitwise multiplication symbol of the elements, σ(·) is the activation function, M a 、M b are the first and second learnable parameter matrices, e k is the text supplementary information representation of the kth scale, and d is the number of matrix columns.

[0059] S4: Obtain the feature matrix of the subgraph to be inferred, the retrieval subgraph set of the previous scale, and the feature matrices of several retrieval subgraphs in the retrieval subgraph set; input the retrieval subgraph set of the previous scale, the feature matrices of several retrieval subgraphs in the retrieval subgraph set, the subgraph to be inferred, the feature matrix of the subgraph to be inferred, and the transformation weight matrix of the current scale into the interactive unit for interactive processing to obtain the interactive feature matrix of the subgraph to be inferred at the current scale.

[0060] In this embodiment, the inference device obtains the feature matrix of the subgraph to be inferred, the retrieval subgraph set of the previous scale, and the feature matrices of several retrieval subgraphs in the retrieval subgraph set.

[0061] To ensure that the subgraph to be inferred can make full use of the graph topology in the current scale, and at the same time absorb important information from the retrieval subgraph set retrieved from the previous scale and several retrieval subgraphs in the retrieval subgraph set, the inference device inputs the retrieval subgraph set of the previous scale, the feature matrix of several retrieval subgraphs in the retrieval subgraph set, the subgraph to be inferred, the feature matrix of the subgraph to be inferred, and the transformation weight matrix of the current scale into the interaction unit for interactive processing to obtain the interactive feature matrix of the subgraph to be inferred at the current scale.

[0062] See also Figure 4 , Figure 4 The flowchart of S4 in the information reasoning method provided in one embodiment of the present application includes steps S41 to S42, which are specifically as follows:

[0063] S41: performing similarity calculation based on the node vectors of several nodes in the subgraph to be inferred, and constructing an intra-scale adjacency matrix; performing similarity calculation based on the node vectors of several nodes in the subgraph to be inferred and a set of retrieval subgraphs in a previous scale, and constructing a cross-scale adjacency matrix.

[0064] In this embodiment, the inference device performs similarity calculation based on the node vectors of several nodes in the subgraph to be inferred, and selects the first several nodes to establish a connection with the currently selected node based on the similarity, which is recorded as Topk. nodes1 The inference device is between the currently selected node and Topk nodes1 Edges are established between the nodes to construct an intra-scale adjacency matrix, wherein the intra-scale adjacency matrix includes similarity vectors between a plurality of nodes in the subgraph to be inferred.

[0065] Since different scales use different node aggregation ratios, there may be differences between scales. The inference device calculates the similarity of the node vectors of several nodes in the subgraph to be inferred and the retrieval subgraph set of the previous scale, and selects the first several nodes to establish a connection with the currently selected node based on the similarity, which is recorded as Topk nodes2 The inference device is between the currently selected node and Topk nodes2 Edges are established between them to construct a cross-scale adjacency matrix, wherein the cross-scale adjacency matrix includes similarity vectors between several nodes in the subgraph to be inferred and several nodes in several retrieval subgraphs.

[0066] S42: Obtain an interactive feature matrix of the subgraph to be inferred at the current scale according to feature matrices of several retrieval subgraphs in the retrieval subgraph set at the previous scale, a feature matrix of the subgraph to be inferred, a transformation weight matrix at the current scale, and a preset multi-scale interactive algorithm.

[0067] In this embodiment, the inference device adopts a graph neural network to obtain the interaction feature matrix of the subgraph to be inferred at the current scale according to the feature matrix of several retrieval subgraphs in the retrieval subgraph set at the previous scale, the feature matrix of the subgraph to be inferred, the transformation weight matrix of the current scale and the preset multi-scale interaction algorithm, wherein the multi-scale interaction algorithm is:

[0068]

[0069] In the formula, is the interactive feature matrix of the k-th scale subgraph to be inferred, σ(·) is the activation function, A intra is the intra-scale adjacency matrix, H t is the feature matrix of the subgraph to be inferred, is the transformation weight matrix of the kth scale, Across is the cross-scale adjacency matrix, is the feature matrix of the rth retrieval subgraph in the k-1th scale retrieval subgraph set, W cross is the preset cross-scale weight matrix.

[0070] S5: Input the zoomed graph of the current scale and the feature matrix of the zoomed graph, the subgraph to be inferred and the interactive feature matrix of the subgraph to be inferred into the graph core retrieval unit to construct a retrieval subgraph, obtain a retrieval subgraph set of the current scale, and repeat the process until a retrieval subgraph set of the last scale is obtained, and obtain retrieval subgraph sets of several scales.

[0071] In this embodiment, the inference device inputs the zoom graph of the current scale and the feature matrix of the zoom graph, the subgraph to be inferred and the interactive feature matrix of the subgraph to be inferred into the graph core retrieval unit to construct a retrieval subgraph, obtain a retrieval subgraph set of the current scale, and repeat the process until a retrieval subgraph set of the last scale is obtained, thereby obtaining retrieval subgraph sets of several scales.

[0072] By adopting graph neural networks and combining transformation weight matrices of different scales to aggregate neighboring node information from different scales, we can make full use of information at different scales and improve the accuracy of retrieval subgraph construction.

[0073] See also Figure 5 , Figure 5 The flowchart of S4 in the information reasoning method provided in one embodiment of the present application includes steps S51 to S52, which are specifically as follows:

[0074] S51: using a random sampling method to obtain a number of candidate subgraphs from the zoomed graph of the current scale, and obtaining an adjacency matrix and a feature matrix of the number of candidate subgraphs of the current scale according to the zoomed graph of the current scale and the feature matrix of the zoomed graph.

[0075] In this embodiment, the inference device adopts a random sampling method to obtain several candidate subgraphs from the zoomed graph of the current scale, and obtains the adjacency matrix and the feature matrix of the several candidate subgraphs of the current scale according to the zoomed graph of the current scale and the feature matrix of the zoomed graph.

[0076] S52: Obtain the adjacency matrix of the subgraph to be inferred as the interactive adjacency matrix of the current scale, and obtain the similarity between the subgraph to be inferred and several candidate subgraphs of the current scale according to the interactive adjacency matrix of the current scale, the interactive feature matrix of the subgraph to be inferred, the adjacency matrices and feature matrices of several candidate subgraphs, and a preset similarity calculation algorithm; determine several retrieval subgraphs from the several candidate subgraphs according to the similarity, and obtain a retrieval subgraph set of the current scale.

[0077] In this embodiment, the inference device obtains the adjacency matrix of the subgraph to be inferred as the interactive adjacency matrix of the current scale, and searches using a random walk graph kernel according to the interactive adjacency matrix of the current scale, the interactive feature matrix of the subgraph to be inferred, the adjacency matrix of several candidate subgraphs, the feature matrix, and a preset similarity calculation algorithm. The structural similarity between the graphs is measured by calculating the number of common paths, and the feature similarity is further evaluated by node attributes to obtain the similarity between the subgraph to be inferred and several candidate subgraphs of the current scale, wherein the similarity calculation algorithm is:

[0078]

[0079] Where simi(·) is the similarity function, is the interaction adjacency matrix of the kth scale, is the feature matrix of the lth candidate subgraph at the kth scale, is the adjacency matrix of the lth candidate subgraph at the kth scale, vec(·) is the matrix function, is the dot product symbol.

[0080] The inference device selects the first several candidate subgraphs with the highest similarity from the several candidate subgraphs as retrieval subgraphs according to the similarity, determines several retrieval subgraphs, and obtains a retrieval subgraph set of the current scale.

[0081] S6: Inputting the subgraph to be inferred and a set of retrieval subgraphs of several scales into the reasoning module for information reasoning to obtain an information reasoning result of the subgraph to be inferred.

[0082] In this embodiment, the reasoning device inputs the subgraph to be reasoned and a set of retrieval subgraphs of several scales into the reasoning module for information reasoning to obtain the information reasoning result of the subgraph to be reasoned.

[0083] Specifically, in order to make full use of the information in several retrieval subgraphs in the retrieval subgraph set, the reasoning device adopts a large language model (LLM) as the reasoning module. The reasoning device inputs the subgraph to be reasoned and the retrieval subgraph set of several scales into the large language model in the form of chain thinking (CoT). This method clearly models the reasoning process and gradually stimulates the large language model, so that the large language model can gradually understand the multi-scale graph structure through predefined reasoning steps, strengthen the output of the large language model, and the reasoning device extracts text information from several retrieval subgraphs in the retrieval subgraph set of several scales, and performs information reasoning on the subgraph to be reasoned and the extracted multi-scale retrieval subgraph text information to obtain the information reasoning result of the subgraph to be reasoned.

[0084] In an optional embodiment, the step S7 is further included: training the multi-scale interaction module. Figure 6 , Figure 6 A flowchart of an information reasoning method provided for another embodiment of the present application includes steps S71 to S74, which are specifically as follows:

[0085] S71: Obtain several sample sub-graphs, feature matrices of the sample sub-graphs, and real label data of several scales of the sample sub-graphs.

[0086] In this embodiment, the inference device obtains several sample subgraphs, feature matrices of the sample subgraphs, and real label data of several scales of the sample subgraphs, wherein the real label data includes real label probability vectors of several categories, and the label probability vectors are used to indicate training of the multi-scale interaction module.

[0087] S72: Inputting the plurality of sample subgraphs into the weight construction unit in the multi-scale interaction module to be trained respectively, and obtaining transformation weight matrices of the plurality of scales of the plurality of sample subgraphs.

[0088] In this embodiment, the inference device inputs several sample subgraphs into the weight construction unit in the multi-scale interaction module to be trained, and obtains transformation weight matrices of several scales of the several sample subgraphs. For the specific embodiment, please refer to step S32, which will not be described here.

[0089] S73: obtaining a set of retrieval subgraphs of several scales corresponding to several sample subgraphs and feature matrices of several retrieval subgraphs in the retrieval subgraph set; inputting a set of retrieval subgraphs of the previous scale of the same sample subgraph, feature matrices of several retrieval subgraphs in the retrieval subgraph set, the sample subgraph, the feature matrix of the sample subgraph and the transformation weight matrix of the current scale into an interaction unit in a multi-scale interaction module to be trained for interaction processing, and obtaining interaction feature matrices of several scales of several sample subgraphs.

[0090] In this embodiment, the inference device obtains a set of retrieval subgraphs of several scales corresponding to the sample subgraphs and feature matrices of several retrieval subgraphs in the set of retrieval subgraphs.

[0091] The inference device inputs the retrieval subgraph set of the previous scale of the same sample subgraph, the feature matrices of several retrieval subgraphs in the retrieval subgraph set, the sample subgraph, the feature matrix of the sample subgraph, and the transformation weight matrix of the current scale into the interactive unit in the multi-scale interactive module to be trained for interactive processing, and obtains the interactive feature matrices of several scales of several sample subgraphs. For the specific embodiment, please refer to steps S4 to S5, which will not be repeated here.

[0092] S74: Inputting the interaction feature matrices of several scales of several sample subgraphs into a preset fully connected layer respectively to obtain predicted label data of several scales of several sample subgraphs; obtaining a cross entropy loss value according to the predicted label data of several scales of several sample subgraphs, the real label data and a preset cross entropy loss algorithm, and training the multi-scale interaction module to be trained according to the cross entropy loss value.

[0093] In this embodiment, the inference device inputs the interaction feature matrices of several scales of several sample subgraphs into a preset fully connected layer respectively to obtain predicted label data of several scales of several sample subgraphs, wherein the predicted label data includes predicted label probability vectors of several categories.

[0094] The inference device obtains a cross entropy loss value according to the predicted label data of several scales of several sample subgraphs, the real label data and a preset cross entropy loss algorithm, and trains the multi-scale interaction module to be trained according to the cross entropy loss value, wherein the cross entropy loss algorithm is:

[0095]

[0096] Where L is the cross entropy loss value, N is the number of sample subgraphs, is the true label data of the kth scale of the i-th sample subgraph, is the predicted label data of the kth scale of the i-th sample sub-graph.

[0097] Please refer to Figure 7 , Figure 7 This is a schematic diagram of the structure of an information reasoning device provided in one embodiment of the present application. The device can implement all or part of the information reasoning device through software, hardware, or a combination of both. The device 7 includes:

[0098] A data acquisition module 71 is used to obtain a knowledge graph, a subgraph to be inferred, and a graph-enhanced language model, wherein the graph-enhanced language model includes a multi-scale transformation module, a multi-scale interaction module, and a reasoning module; the multi-scale interaction module includes a matrix construction unit, an interaction unit, and a graph core retrieval unit;

[0099] A multi-scale feature extraction module 72, used to input the knowledge graph into the multi-scale transformation module for transformation processing and feature extraction, and obtain scaling graphs of several scales and feature matrices of the scaling graphs;

[0100] A transformation weight matrix construction module 73 is used to input the subgraph to be inferred into the matrix construction unit to construct a transformation weight matrix, and obtain transformation weight matrices of several scales;

[0101] The multi-scale interaction processing module 74 is used to obtain the feature matrix of the subgraph to be inferred, the retrieval subgraph set of the previous scale, and the feature matrices of several retrieval subgraphs in the retrieval subgraph set; the retrieval subgraph set of the previous scale, the feature matrices of several retrieval subgraphs in the retrieval subgraph set, the subgraph to be inferred, the feature matrix of the subgraph to be inferred, and the transformation weight matrix of the current scale are input into the interaction unit for interaction processing to obtain the interaction feature matrix of the subgraph to be inferred at the current scale;

[0102] The retrieval subgraph construction module 75 is used to input the zoomed graph of the current scale and the feature matrix of the zoomed graph, the subgraph to be inferred and the interactive feature matrix of the subgraph to be inferred into the graph core retrieval unit to construct the retrieval subgraph, obtain the retrieval subgraph set of the current scale, and repeat the process until the retrieval subgraph set of the last scale is obtained, and obtain the retrieval subgraph sets of several scales;

[0103] The subgraph information reasoning module 76 is used to input the subgraph to be reasoned and a set of search subgraphs of several scales into the reasoning module for information reasoning to obtain the information reasoning result of the subgraph to be reasoned.

[0104] In an embodiment of the present application, a knowledge graph, a subgraph to be inferred, and a graph-enhanced language model are obtained through a data acquisition module, wherein the graph-enhanced language model includes a multi-scale transformation module, a multi-scale interaction module, and a reasoning module; the multi-scale interaction module includes a matrix construction unit, an interaction unit, and a graph core retrieval unit; through a multi-scale feature extraction module, the knowledge graph is input into the multi-scale transformation module for transformation processing and feature extraction, and a scaling graph of several scales and a feature matrix of the scaling graph are obtained; through a transformation weight matrix construction module, the subgraph to be inferred is input into the matrix construction unit for transformation weight matrix construction, and a transformation weight matrix of several scales is obtained; through a multi-scale interaction processing module, a feature matrix of the subgraph to be inferred, a retrieval subgraph set of the previous scale, and feature matrices of several retrieval subgraphs in the retrieval subgraph set are obtained. Matrix; input the retrieval subgraph set of the previous scale, the feature matrices of several retrieval subgraphs in the retrieval subgraph set, the subgraph to be inferred, the feature matrix of the subgraph to be inferred and the transformation weight matrix of the current scale into the interaction unit for interactive processing to obtain the interactive feature matrix of the subgraph to be inferred at the current scale; through the retrieval subgraph construction module, input the zoomed graph of the current scale and the feature matrix of the zoomed graph, the subgraph to be inferred and the interactive feature matrix of the subgraph to be inferred into the graph core retrieval unit for retrieval subgraph construction to obtain the retrieval subgraph set of the current scale, and repeat the execution until the retrieval subgraph set of the last scale is obtained, and the retrieval subgraph sets of several scales are obtained; through the subgraph information reasoning module, input the subgraph to be inferred and the retrieval subgraph sets of several scales into the reasoning module for information reasoning to obtain the information reasoning result of the subgraph to be inferred. The knowledge graph is transformed at multiple scales to construct a multi-scale zoom graph of the knowledge graph. The subgraph to be inferred and the multi-scale transformation weight matrix constructed based on the subgraph to be inferred are combined for multi-scale interaction. The information of neighboring nodes from different scales is aggregated, and the information of different scales is fully utilized to construct a more accurate retrieval subgraph for graph information reasoning, thereby improving the accuracy of accurate information reasoning on the graph.

[0105] Please refer to Figure 8 , Figure 8 The computer device 8 is a schematic diagram of a structure of a computer device provided in an embodiment of the present application. The computer device 8 includes: a processor 81, a memory 82, and a computer program 83 stored in the memory 82 and executable on the processor 81; the computer device may store multiple instructions, which are suitable for being loaded and executed by the processor 81. Figures 1 to 8 The method steps shown in the figure can be found in the specific execution process. Figures 1 to 8 The specific description shown will not be repeated here.

[0106] Among them, the processor 81 may include one or more processing cores. The processor 81 uses various interfaces and lines to connect various parts in the server, and executes various functions and processes data of the information reasoning device 7 by running or executing instructions, programs, code sets or instruction sets stored in the memory 82, and calling the data in the memory 82. Optionally, the processor 81 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programble Logic Array, PLA). The processor 81 can integrate one or more combinations of a central processing unit 81 (Central Processing Unit, CPU), a graphics processor 81 (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the touch display; the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 81, and it can be implemented by a single chip.

[0107] Among them, the memory 82 may include a random access memory 82 (Random Access Memory, RAM), and may also include a read-only memory 82 (Read-Only Memory). Optionally, the memory 82 includes a non-transitory computer-readable storage medium. The memory 82 can be used to store instructions, programs, codes, code sets or instruction sets. The memory 82 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch instructions, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area may store data involved in the above-mentioned various method embodiments, etc. The memory 82 may also be optionally at least one storage device located away from the aforementioned processor 81.

[0108] The present application also provides a storage medium that can store multiple instructions, which are suitable for the processor to load and execute the above-mentioned Figures 1 to 8 The method steps shown in the figure can be found in the specific execution process. Figures 1 to 8 The specific description shown will not be repeated here.

[0109] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0110] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0111] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraint algorithm of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0112] In the embodiments provided by the present invention, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0113] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0114] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0115] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc.

[0116] The present invention is not limited to the above-mentioned embodiments. If various changes or modifications to the present invention do not depart from the spirit and scope of the present invention, and if these changes and modifications fall within the scope of the claims and equivalent technologies of the present invention, the present invention is also intended to include these changes and modifications.

Claims

1. An information reasoning method, characterized in that: The method comprises the following steps: Obtaining a knowledge graph, a subgraph to be inferred, and a graph-enhanced language model, wherein the graph-enhanced language model includes a multi-scale transformation module, a multi-scale interaction module, and a reasoning module; the multi-scale interaction module includes a matrix construction unit, an interaction unit, and a graph core retrieval unit; Inputting the knowledge graph into the multi-scale transformation module for transformation processing and feature extraction to obtain scaling graphs of several scales and feature matrices of the scaling graphs; Inputting the subgraph to be inferred into the matrix construction unit to construct a transformation weight matrix to obtain transformation weight matrices of several scales; Obtaining a feature matrix of the subgraph to be inferred, a retrieval subgraph set of a previous scale, and feature matrices of several retrieval subgraphs in the retrieval subgraph set; inputting the retrieval subgraph set of a previous scale, feature matrices of several retrieval subgraphs in the retrieval subgraph set, the subgraph to be inferred, the feature matrix of the subgraph to be inferred, and a transformation weight matrix of the current scale into the interactive unit for interactive processing, and obtaining an interactive feature matrix of the subgraph to be inferred at the current scale; Inputting the zoomed graph of the current scale and the feature matrix of the zoomed graph, the subgraph to be inferred and the interactive feature matrix of the subgraph to be inferred into the graph core retrieval unit to construct a retrieval subgraph, obtaining a retrieval subgraph set of the current scale, and repeating the process until a retrieval subgraph set of the last scale is obtained, thereby obtaining retrieval subgraph sets of several scales; The subgraph to be inferred and a set of retrieval subgraphs of several scales are input into the reasoning module for information reasoning to obtain the information reasoning result of the subgraph to be inferred.

2. The information reasoning method according to claim 1, characterized in that: The step of inputting the knowledge graph into the multi-scale transformation module for transformation processing and feature extraction to obtain scaling graphs of several scales and a feature matrix of the scaling graphs includes the following steps: Aggregating and classifying the nodes in the knowledge graph according to the preset scaling factors of several scales, converting the knowledge graph into scaling graphs of several scales, and obtaining scaling graphs of several scales, wherein the scaling graph includes several nodes, and the nodes include important nodes and non-important nodes; Convert text attributes of several nodes in zoom graphs of several scales, and construct important node feature matrices and non-important node feature matrices of the zoom graphs of several scales; Obtain node matching relationship matrices of zoom graphs of several scales, and obtain feature aggregation matrices of zoom graphs of several scales according to the node matching relationship matrices of zoom graphs of several scales, important node feature matrices, unimportant node feature matrices, and a preset feature aggregation algorithm as feature matrices of the zoom graphs, wherein the feature aggregation algorithm is: Where M k is the node matching relationship matrix of the kth scale, is the node matching relationship vector between the ith node and the jth node in the node matching relationship matrix of the kth scale, is the node matching relationship matrix M k The degree vector of the i-th node in k M is the node matching relationship matrix k The degree matrix of is the feature matrix of the non-important nodes of the scaled graph at the kth scale, X ′ k Based on matching relationship The cumulative feature matrix of is the important node feature matrix of the k-th scaled graph, is the similarity feature matrix between the feature matrix of important nodes and the feature matrix of unimportant nodes in the scaled graph of the kth scale, I is the unit matrix, is the feature aggregation matrix of the k-th scaled graph.

3. The information reasoning method according to claim 2, characterized in that: The step of inputting the subgraph to be inferred into the matrix construction unit to construct a transformation weight matrix to obtain transformation weight matrices of several scales comprises the following steps: Extracting the text supplementary information of the subgraph to be inferred, performing multi-scale embedding processing on the text supplementary information, and obtaining representations of the text supplementary information at several scales; According to the text supplementary information representation of several scales and the preset transformation weight matrix calculation algorithm, the transformation weight matrix of several scales is obtained, wherein the transformation weight matrix calculation algorithm is: In the formula, is the transformation weight matrix of the kth scale, W intra is the initial weight matrix, ⊙ is the bitwise multiplication symbol of the elements, σ(·) is the activation function, M a 、M b are the first and second learnable parameter matrices, e k is the text supplementary information representation of the kth scale, and d is the number of matrix columns.

4. The information reasoning method according to claim 2 or 3, characterized in that: The step of inputting the retrieval subgraph set of the previous scale, the feature matrices of several retrieval subgraphs in the retrieval subgraph set, the subgraph to be inferred, the feature matrix of the subgraph to be inferred, and the transformation weight matrix of the current scale into the multi-scale interaction module for interactive processing to obtain the interactive feature matrix of the subgraph to be inferred at the current scale includes the following steps: Perform similarity calculations on the node vectors of several nodes in the subgraph to be inferred, and construct an intra-scale adjacency matrix; perform similarity calculations on the node vectors of several nodes in the subgraph to be inferred and several retrieval subgraphs in the retrieval subgraph set of the previous scale, and construct a cross-scale adjacency matrix, wherein the intra-scale adjacency matrix includes similarity vectors between several nodes in the subgraph to be inferred; and the cross-scale adjacency matrix includes similarity vectors between several nodes in the subgraph to be inferred and several nodes in several retrieval subgraphs; According to the feature matrices of several retrieval subgraphs in the retrieval subgraph set of the previous scale, the feature matrix of the subgraph to be inferred, the transformation weight matrix of the current scale and the preset multi-scale interaction algorithm, the interaction feature matrix of the subgraph to be inferred at the current scale is obtained, wherein the multi-scale interaction algorithm is: In the formula, is the interactive feature matrix of the k-th scale subgraph to be inferred, σ(·) is the activation function, A intra is the intra-scale adjacency matrix, H t is the feature matrix of the subgraph to be inferred, is the transformation weight matrix of the kth scale, A cross is the cross-scale adjacency matrix, is the feature matrix of the rth retrieval subgraph in the retrieval subgraph set of the k-1th scale, W cross is the preset cross-scale weight matrix.

5. The information inference method according to claim 4, characterized in that: The step of inputting the zoomed graph of the current scale and the feature matrix of the zoomed graph, the subgraph to be inferred and the interactive feature matrix of the subgraph to be inferred into the graph core retrieval unit to construct a retrieval subgraph, and obtaining a retrieval subgraph set of the current scale includes the following steps: A random sampling method is used to obtain a plurality of candidate subgraphs from the zoomed graph of the current scale, and an adjacency matrix and a feature matrix of the plurality of candidate subgraphs of the current scale are obtained according to the zoomed graph of the current scale and the feature matrix of the zoomed graph; The adjacency matrix of the subgraph to be inferred is obtained as the interactive adjacency matrix of the current scale. According to the interactive adjacency matrix of the current scale, the interactive feature matrix of the subgraph to be inferred, the adjacency matrix of several candidate subgraphs, the feature matrix and the preset similarity calculation algorithm, the similarity between the subgraph to be inferred and several candidate subgraphs of the current scale is obtained. According to the similarity, several retrieval subgraphs are determined from the several candidate subgraphs to obtain a retrieval subgraph set of the current scale, wherein the similarity calculation algorithm is: Where simi(·) is the similarity function, is the interaction adjacency matrix of the kth scale, is the feature matrix of the lth candidate subgraph at the kth scale, is the adjacency matrix of the lth candidate subgraph at the kth scale, vec(·) is the matrix function, is the dot product symbol.

6. The information reasoning method according to claim 5, characterized in that: It also includes the steps of: training the multi-scale interaction module; The training of the multi-scale interaction module comprises the steps of: Obtaining a plurality of sample subgraphs, feature matrices of the sample subgraphs, and true label data of a plurality of scales of the sample subgraphs; Inputting several sample subgraphs into the weight construction unit in the multi-scale interaction module to be trained respectively, and obtaining transformation weight matrices of several scales of the several sample subgraphs; Obtaining a retrieval subgraph set of several scales corresponding to several sample subgraphs and feature matrices of several retrieval subgraphs in the retrieval subgraph set; inputting a retrieval subgraph set of the previous scale of the same sample subgraph, feature matrices of several retrieval subgraphs in the retrieval subgraph set, the sample subgraph, the feature matrix of the sample subgraph and the transformation weight matrix of the current scale into an interaction unit in a multi-scale interaction module to be trained for interaction processing, and obtaining interaction feature matrices of several scales of several sample subgraphs; The interactive feature matrices of several scales of several sample subgraphs are respectively input into a preset fully connected layer to obtain predicted label data of several scales of several sample subgraphs; a cross entropy loss value is obtained according to the predicted label data of several scales of several sample subgraphs, the real label data and a preset cross entropy loss algorithm, and the multi-scale interaction module to be trained is trained according to the cross entropy loss value, wherein the cross entropy loss algorithm is: Where L is the cross entropy loss value, N is the number of sample subgraphs, is the true label data of the kth scale of the i-th sample subgraph, is the predicted label data of the kth scale of the i-th sample sub-graph.

7. An information reasoning device, characterized in that: include: A data acquisition module, used to obtain a knowledge graph, a subgraph to be inferred, and a graph-enhanced language model, wherein the graph-enhanced language model includes a multi-scale transformation module, a multi-scale interaction module, and a reasoning module; the multi-scale interaction module includes a matrix construction unit, an interaction unit, and a graph core retrieval unit; A multi-scale feature extraction module, used to input the knowledge graph into the multi-scale transformation module for transformation processing and feature extraction, and obtain scaling graphs of several scales and feature matrices of the scaling graphs; A transformation weight matrix construction module, used for inputting the subgraph to be inferred into the matrix construction unit to construct a transformation weight matrix, and obtaining transformation weight matrices of several scales; A multi-scale interactive processing module is used to obtain the feature matrix of the subgraph to be inferred, the retrieval subgraph set of the previous scale, and the feature matrices of several retrieval subgraphs in the retrieval subgraph set; the retrieval subgraph set of the previous scale, the feature matrices of several retrieval subgraphs in the retrieval subgraph set, the subgraph to be inferred, the feature matrix of the subgraph to be inferred, and the transformation weight matrix of the current scale are input into the interactive unit for interactive processing to obtain the interactive feature matrix of the subgraph to be inferred at the current scale; A retrieval subgraph construction module is used to input the zoomed graph of the current scale and the feature matrix of the zoomed graph, the subgraph to be inferred and the interactive feature matrix of the subgraph to be inferred into the graph core retrieval unit to construct a retrieval subgraph, obtain a retrieval subgraph set of the current scale, and repeat the process until a retrieval subgraph set of the last scale is obtained, thereby obtaining retrieval subgraph sets of several scales; The subgraph information reasoning module is used to input the subgraph to be reasoned and a set of search subgraphs of several scales into the reasoning module for information reasoning to obtain the information reasoning result of the subgraph to be reasoned.

8. A computer device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor; when the computer program is executed by the processor, the steps of the information reasoning method according to any one of claims 1 to 6 are implemented.

9. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the information reasoning method according to any one of claims 1 to 6 are implemented.

Images