A relationship-guided full-induction multi-modal knowledge graph reasoning method and system

Abstract

Embodiments of the present application disclose a relationship-guided full-inductive multi-modal knowledge graph reasoning method and system. The method comprises: generating multi-modal semantic labels for a target entity and extracting its local structure context in a reasoning graph; attention filtering of the semantic labels is guided by a query relationship to construct relationship-guided semantic features; the local structure context is encoded to obtain structure features; semantic and structure features are fused through an adaptive gating mechanism to generate an initial representation of the entity; global message passing is performed on a message passing network to enhance the representation; and finally, the reasoning is completed by calculating the triple score based on the entity representation and the relationship representation. Through relationship-guided semantic filtering and adaptive gating fusion, the present application effectively suppresses multi-modal semantic noise, maintains structure generalization capability, and significantly improves the accuracy and robustness of knowledge graph reasoning under full-inductive settings.

Description

Technical Field

[0001] This invention relates to the field of computer knowledge graph technology, and in particular to a relation-guided fully inductive multimodal knowledge graph reasoning method and system. Background Technology

[0002] Knowledge graphs represent real-world knowledge in a structured manner as triples (head entity, relation, tail entity) and have been widely applied in information retrieval, intelligent question answering, and recommendation systems. However, new entities are constantly emerging in the real world, and traditional inductive reasoning methods struggle to handle entities not seen during training. Existing technologies employ inductive knowledge graph reasoning to predict missing triples between new entities not observed during training. In this task setting, the entity sets in the training graph and the reasoning graph are completely disjoint, sharing only the relation set.

[0003] However, inductive knowledge graph reasoning relies solely on graph structure information, making it difficult to distinguish the true semantic differences of entities in structurally isomorphic scenarios, leading to ambiguity in the reasoning results. To alleviate this problem, introducing multimodal information is considered a feasible approach. By introducing multimodal features, individualized characteristics can be provided for unseen entities. However, existing multimodal knowledge graph reasoning techniques still have the following shortcomings under the fully inductive task setting: First, multimodal semantic noise is severe. In the fully inductive setting, entities are completely invisible during the training phase, and the model cannot learn specific multimodal representations for specific entities. It can only rely on general pre-trained models to extract modal features. These features usually contain a large amount of semantic information unrelated to the target relationship. Directly participating in inference can easily introduce noise and reduce inference accuracy.

[0004] Second, modality dominance leads to a degradation in structural generalization ability. Multimodal features are continuous and high-dimensional, making them easier for models to utilize compared to discrete and sparse graph structural information. During training, models often over-rely on multimodal information to achieve rapid convergence, neglecting to fully learn structural reasoning capabilities. When the multimodal distribution shifts during the inference phase, model performance deteriorates significantly, even falling short of methods that rely solely on structural information. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a relation-guided fully inductive multimodal knowledge graph reasoning method and system, which utilizes relation-guided semantic filtering and adaptive gating fusion to obtain highly accurate and robust multimodal knowledge graph reasoning results under a fully inductive setting. On the one hand, a relation-guided, fully inductive, multimodal knowledge graph reasoning method is provided, the method comprising: The system acquires image data and text description data corresponding to the target entity to be processed in the knowledge graph and generates structured semantic tags. The knowledge graph is constructed by entity recognition and relation extraction of knowledge. The knowledge is related to the question answering system and semantic search. The entity is text data obtained from natural language text using named entity recognition tools. For the target entity to be processed in the knowledge graph reasoning stage, extract its local structural context in the current reasoning graph; Guided by query relationships, attention filtering is performed on the structured semantic tags to construct a relationship-guided semantic feature vector; Encode the local structural context to obtain local structural features; An initial representation of the target entity is generated by fusing the relation-guided semantic features and the local structural features through an adaptive gating mechanism. Using the initial representation as node features, global message passing is performed on the message passing network to obtain the enhanced final entity representation; Based on the final entity representation and query relation representation, the probability of the target triple is calculated to complete the reasoning.

[0006] According to the present invention, a relation-guided fully inductive multimodal knowledge graph reasoning method is provided, wherein generating a structured semantic tag list includes: for any entity The image associated with it and text description The input is fed into a large language model, and a list of semantic tags for the entity is generated through prompt word engineering; According to the present invention, a relation-guided, fully inductive, multimodal knowledge graph reasoning method is provided, wherein the local structural context includes: a target entity Construct a set of local relation sequences based on the central one-hop neighbor nodes and their connections.

[0007] According to the present invention, a relation-guided fully inductive multimodal knowledge graph reasoning method is provided, wherein constructing the relation-guided semantic feature vector includes: utilizing the embedding vector of the query relation. As a query vector, the list of semantic tags Each label in the algorithm is mapped to an embedding vector as a key and value. Attention weights are calculated and aggregated to generate a semantic feature vector. :

[0008]

[0009] in, This is the embedding vector for the query relation. For the first Embedding vectors of each label; The projection matrix is ​​learnable; This is the dimension scaling factor; The normalized exponential function is used to map the calculated similarity score to... arrive The attention weights between them are determined, and the sum of the weights of all labels is guaranteed to be equal to the sum of the weights of all labels. ; For the first The semantic importance score of each label relative to the current relationship.

[0010] According to the relation-guided fully inductive multimodal knowledge graph reasoning method provided by the present invention, encoding the local structural context to obtain local structural features includes: setting a set of local relation sequences. Each relation in the model is mapped to an embedding vector, which is then transformed using a multilayer perceptron and aggregated using average pooling to obtain local structural feature vectors. :

[0011] in, This represents the pooling operation, used to aggregate multiple vectors into a single vector of fixed length. This represents a multilayer perceptron, used to perform feature transformation and nonlinear mapping on the relational features of the input; Represents a set of local relation sequences The embedding vector of any relation in the dataset; This refers to the set of local relation sequences of the target entity extracted in the aforementioned steps.

[0012] According to the relation-guided fully inductive multimodal knowledge graph reasoning method provided by the present invention, the generation of the initial representation of the entity includes: introducing an adaptive gating mechanism to dynamically adjust the contribution ratio of semantic features and structural features to generate an initial node representation.

[0013]

[0014] in, This represents a vector concatenation operation. It is the Sigmoid activation function. Here is the weight matrix of the gated network. These are the bias parameters for the gating system. It is an adaptive gating coefficient.

[0015] According to the relation-guided fully inductive multimodal knowledge graph reasoning method provided by the present invention, the step of performing global message passing on a message passing network to obtain the enhanced final entity representation includes: using generated fusion features As the initial features of the nodes, graph neural networks are used for... Layer feature updates yield the final entity representation. :

[0016] in, Represents the target entity In the The updated feature representation of the layer network; This represents the node feature update function of a graph neural network; Represents the target entity In the Layer feature representation; Represents the neighborhood feature aggregation function; Representing neighboring nodes In the Layer feature representation; Represents the target entity The set of one-hop neighbor nodes in a message-passing network; Represents the target entity The The initial feature representation of the layer is the fusion feature generated in step S5.

[0017] According to the relation-guided fully inductive multimodal knowledge graph reasoning method provided by the present invention, the calculation of the probability of the target triple includes: using RotatE as a scoring function, for a given query Calculate candidate tail entities Score:

[0018] in, Indicates the entity by head ,relation and candidate tail entities The score for the formation of the triple; Represents the embedding vector corresponding to a given head entity; This represents the embedding vector corresponding to the query relationship; This represents the embedding vector corresponding to the candidate tail entity; Represents head entity The final entity representation vector obtained after the aforementioned global message passing; Represents candidate tail entities The final entity representation vector obtained after the aforementioned global message passing; It represents the Hadamardi (or Hadama) stack; Norms are used to measure the distance between vectors.

[0019] On the other hand, a relation-guided, fully inductive, multimodal knowledge graph reasoning system is provided, the system comprising: The semantic tag generation module acquires image data and text description data corresponding to the target entity to be processed in the knowledge graph, and generates structured semantic tags; the knowledge graph is constructed by entity recognition and relation extraction of knowledge, the knowledge is related knowledge in question answering system and semantic search, and the entity is text data obtained from natural language text using named entity recognition tools; The structural context extraction module extracts the local structural context of the target entity to be processed in the knowledge graph reasoning stage within the current reasoning graph. The relation-guided semantic feature construction module, guided by query relations, performs attention filtering on the structured semantic tags to construct a relation-guided semantic feature vector; The local structural feature encoding module encodes the local structural context to obtain local structural features; The gating fusion module, through an adaptive gating mechanism, fuses the relation-guided semantic features and the local structural features to generate an initial representation of the target entity; The entity representation enhancement module uses the initial representation as node features and performs global message passing on the message passing network to obtain the enhanced final entity representation. The reasoning scoring module calculates the probability of the target triplet being true based on the final entity representation and the query relationship representation, thus completing the reasoning.

[0020] In another aspect, a storage medium is also provided for non-transitory storage of computer-readable instructions, wherein when the non-transitory computer-readable instructions are executed by a computer, the method described in the first aspect is performed.

[0021] The above technical solution has the following advantages or beneficial effects: This invention employs a relation-guided attention mechanism, using the current query relation as a guide to dynamically filter key information from discrete semantic tags generated by a large language model. Compared to directly using multimodal features extracted from a general pre-trained model, this invention retains only semantic tags highly relevant to the target relation for subsequent reasoning, significantly reducing interference from irrelevant semantic information and improving the effectiveness of multimodal information in knowledge graph reasoning tasks.

[0022] This invention employs an adaptive gating fusion mechanism to dynamically adjust the contribution ratio of semantic and structural features. The gating coefficients can adaptively adjust with the input: when the multimodal distribution shifts or noise is high during the inference stage, the model automatically reduces its dependence on semantic features and enhances its focus on structural features; when the quality of multimodal information is high, the model can fully utilize semantic information to improve entity discrimination. This effectively avoids the degradation of structural reasoning ability caused by over-reliance on multimodal information, ensuring the model's stability and generalization ability under different data distributions.

[0023] This invention utilizes a joint module method of "semantic denoising + adaptive fusion + structural enhancement" to overcome the limitation of ambiguity in pure structural reasoning in isomorphic scenarios, while reducing the negative interference introduced by multimodal approaches. In the scenario of fully inductive knowledge graph reasoning, it significantly improves the accuracy and robustness of link prediction. Attached Figure Description

[0024] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0025] Figure 1 The flowchart shows a relation-guided fully inductive multimodal knowledge graph reasoning method provided in Embodiment 1 of the present invention. Figure 2 This is a schematic diagram of the architecture of a relation-guided, fully inductive multimodal knowledge graph reasoning method provided in Embodiment 1 of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present invention.

[0027] Before introducing the technical solution of this application, the terms used in this application will be explained first: Multimodal knowledge graph: Based on the traditional triplet knowledge graph, unstructured modal data is introduced as the attribute or association information of the entity in the knowledge graph.

[0028] Total inductive reasoning: refers to a task setting where the entities in the reasoning phase never appear in the training phase, and the reasoning graph and the training graph only share relation types, while the entity sets are completely disjoint.

[0029] Structural isomorphism: In pure structural reasoning, if the one-hop or multi-hop neighbor relationships around two different entities are very similar, a model that relies solely on graph structure will generate almost identical representation vectors, causing the model to be unable to distinguish between the two entities, thus creating ambiguity.

[0030] Generalization ability: refers to the ability of a machine learning model to adapt to data that has not been seen during training.

[0031] Example 1 This embodiment provides a relation-guided, fully inductive, multimodal knowledge graph reasoning method; S1: Obtain image data and text description data corresponding to the target entity to be processed in the knowledge graph, and generate structured semantic tags; the knowledge graph is constructed by entity recognition and relation extraction of knowledge, the knowledge is related knowledge in question answering system and semantic search, and the entity is text data obtained from natural language text using named entity recognition tools; For entities in a knowledge graph, multimodal data of entity associations is obtained. A Large Language Model (LLM) is used as a general tag generator to transform unstructured multimodal data into a structured list of semantic tags. Specifically, this includes processing the target entity... The image associated with it and text description The input is fed into a large language model, and a list of semantic tags for the entity is generated through prompt word engineering. ,in The number of labels is denoted by . For example, taking the movie entity "Forrest Gump" in the FB15k-237 dataset, its associated multimodal data includes promotional poster images for the movie and the text description "A 1994 American epic romantic comedy film directed by Robert Zemeckis". The large language model will extract and generate a series of discrete phrases such as {"movie", "comedy", "Robert Zemeckis", "USA", "1994"} as a list of semantic labels for this entity.

[0032] S2: Under the full inductive setting, for the target entity to be processed, extract its local topological structure in the inference graph as the structural context.

[0033] Specifically, this includes the target entity to be processed. Traverse its one-hop neighbor nodes and their connections in the current graph structure to construct a set of local relation sequences. ,in This indicates a relationship pointing to that entity. This indicates the relationship that the entity points to.

[0034] S3: Guided by the query relationship, perform attention filtering on the structured semantic tags to construct a relationship-guided semantic feature vector; To address the issue of high noise in general multimodal features, a relation-guided attention mechanism is employed. Guided by the current query relation, key information is filtered from semantic labels. Specifically, this involves utilizing the embedding vector of the query relation. As a query vector, the entity list of semantic tags Each label in the vector is mapped to an embedding vector as both a key and a value. Attention weights are calculated and aggregated to generate a semantic feature vector. :

[0035]

[0036] in, This is the embedding vector for the query relation. For the first Embedding vectors of each label; The projection matrix is ​​learnable; This is the dimension scaling factor; The normalized exponential function is used to map the calculated similarity score to... arrive The attention weights between them are determined, and the sum of the weights of all labels is guaranteed to be equal to the sum of the weights of all labels. ; For the first The semantic importance score of each label relative to the current relationship.

[0037] S4: Encode the local structural context to obtain local structural features; To capture the local structural attributes of entities, the local relation sequences extracted by S2 are encoded using an MLP Pooler. Specifically, this involves encoding the set of local relation sequences... Each relation in the model is mapped to an embedding vector, which is then transformed using a multilayer perceptron and aggregated using average pooling to obtain local structural feature vectors. :

[0038] in, This represents the pooling operation, used to aggregate multiple vectors into a single vector of fixed length. This represents a multilayer perceptron, used to perform feature transformation and nonlinear mapping on the relational features of the input; Represents a set of local relation sequences The embedding vector of any relation in the dataset; This is the set of local relation sequences extracted from the target entity in the preceding steps. This feature reflects the local connection pattern of the target entity in the graph.

[0039] S5: By using an adaptive gating mechanism, the semantic features guided by the relationship and the local structural features are fused to generate an initial representation of the entity; To address the degradation of structural generalization ability caused by modality dominance, an adaptive gating mechanism is introduced to dynamically adjust the contribution ratio of semantic features and structural features to generate initial node representations. Specifically, this includes...

[0040]

[0041] in, This represents a vector concatenation operation. The activation function is Sigmoid. This gated network mainly consists of a feature concatenation layer, a fully connected layer, and an activation layer. Its specific processing flow is as follows: First, the semantic feature vectors with consistent dimensions are concatenated through the feature concatenation layer. With local structural feature vectors The features are concatenated along the feature dimension to obtain a high-dimensional joint feature vector; then, this joint feature vector is input into a fully connected layer, through a learnable weight matrix. Perform linear projection mapping and add bias parameters Finally, after a nonlinear transformation through a Sigmoid activation layer, the output is mapped to... arrive The interval is used to obtain the adaptive gating coefficient. .

[0042] For adaptive gating coefficients: when the modal distribution shifts or noise is high during the inference phase, the model can reduce the gating coefficients. The value automatically reduces the dependence on semantic features and enhances the focus on structural features, thereby ensuring robustness.

[0043] S6: Using the initial representation as node features, perform global message passing on the message passing network to obtain the enhanced final entity representation; Fusion features generated by S5 As initial features of nodes, message passing is performed across the entire graph to capture global structural dependencies. First, a message passing network is constructed, which involves acquiring all known triplet facts from the inference phase, using target entities appearing in the inference phase as nodes, and known relationships between entities as edges, to build a network structure reflecting the topological connections between target entities. Then, a graph neural network is used on this message passing network for... Layer feature updates yield the final entity representation. :

[0044] in, Represents the target entity In the The updated feature representation of the layer network; This represents the node feature update function of a graph neural network; Represents the target entity In the Layer feature representation; Represents the neighborhood feature aggregation function; Representing neighboring nodes In the Layer feature representation; Represents the target entity The set of one-hop neighbor nodes in a message-passing network; Represents the target entity The The initial feature representation of the layer is the fused feature generated in step S5. After global enhancement, the entity representation simultaneously contains individual-differentiated multimodal semantic information and global topological structure information.

[0045] S7: Based on the final entity representation and query relationship representation, calculate the probability of the target triplet being true, and complete the reasoning; Using the finally learned entity embeddings and relation embeddings, the probability of a triple being true is calculated based on a knowledge graph scoring function. Specifically, RotatE is used as the scoring function for a given query. Calculate candidate tail entities Score:

[0046] in, Indicates the entity by head ,relation and candidate tail entities The score for the formation of the triple; Represents the embedding vector corresponding to a given head entity; This represents the embedding vector corresponding to the query relationship; This represents the embedding vector corresponding to the candidate tail entity; Represents head entity The final entity representation vector obtained after the aforementioned global message passing; Represents candidate tail entities The final entity representation vector obtained after the aforementioned global message passing; It represents the Hadamardi (or Hadama) stack; Norms are used to measure the distance between vectors.

[0047] In terms of experiments, on the structurally sparse WN18RR dataset, this invention improves Hits@10 to 89.9%, higher than the baseline model GraIL's 73.4%, solving the problem of introducing a large amount of noise by directly using pre-trained features, thus reducing inference accuracy. On the structurally rich but modal noise-sensitive FB15k-237 dataset, this invention improves MMR to 0.7351 and hits@10 to 93.12% compared to the baseline model GraIL, verifying the structural generalization ability of this invention under knowledge graphs of different densities.

[0048] Example 2 The purpose of this embodiment is to provide a relation-guided, fully inductive, multimodal knowledge graph reasoning system.

[0049] A relation-guided, fully inductive, multimodal knowledge graph reasoning system includes: The semantic tag generation module acquires image data and text description data corresponding to target entities in the knowledge graph and generates structured semantic tags. The knowledge graph is constructed by entity recognition and relation extraction of knowledge. The knowledge is related to question answering systems and semantic search. The entities are text data obtained from natural language text using named entity recognition tools. The structural context extraction module extracts the local structural context of target entities that have not appeared in the knowledge graph reasoning stage in the current reasoning graph. The relation-guided semantic feature construction module, guided by query relations, performs attention filtering on the structured semantic tags to construct a relation-guided semantic feature vector; The local structural feature encoding module encodes the local structural context to obtain local structural features; The gating fusion module, through an adaptive gating mechanism, fuses the relation-guided semantic features and the local structural features to generate an initial representation of the target entity; The entity representation enhancement module uses the initial representation as node features and performs global message passing on the message passing network to obtain the enhanced final entity representation. The reasoning scoring module calculates the probability of the target triplet being true based on the final entity representation and the query relationship representation, thus completing the reasoning.

[0050] Example 3 This embodiment also provides a computer-readable storage medium for storing computer instructions, which, when executed by a processor, complete the method described in Embodiment 1.

[0051] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A relation-guided, fully inductive, multimodal knowledge graph reasoning method, characterized in that, The method includes: The system acquires image data and text description data corresponding to the target entity to be processed in the knowledge graph and generates structured semantic tags. The knowledge graph is constructed by entity recognition and relation extraction of knowledge. The knowledge is related to the question answering system and semantic search. The entity is text data obtained from natural language text using named entity recognition tools. For the target entity to be processed in the knowledge graph reasoning stage, extract its local structural context in the current reasoning graph; Guided by query relationships, attention filtering is performed on the structured semantic tags to construct a relationship-guided semantic feature vector; Encode the local structural context to obtain local structural features; An initial representation of the target entity is generated by fusing the relation-guided semantic features and the local structural features through an adaptive gating mechanism. Using the initial representation as node features, global message passing is performed on the message passing network to obtain the enhanced final entity representation; Based on the final entity representation and query relation representation, the probability of the target triple is calculated to complete the reasoning.

2. The relation-guided fully inductive multimodal knowledge graph reasoning method according to claim 1, characterized in that, The generated list of structured semantic tags includes: for any entity The image associated with it and text description The input is fed into a large language model, and a list of semantic tags for the entity is generated through prompt word engineering.

3. The relation-guided fully inductive multimodal knowledge graph reasoning method according to claim 1, characterized in that, The local structural context includes: the target entity Construct a set of local relation sequences based on the central one-hop neighbor nodes and their connections.

4. The relation-guided fully inductive multimodal knowledge graph reasoning method according to claim 1, characterized in that, The semantic feature vector guided by the constructed relationship includes: an embedding vector utilizing the query relationship. As a query vector, the list of semantic tags Each label in the algorithm is mapped to an embedding vector as a key and value. Attention weights are calculated and aggregated to generate a semantic feature vector. : in, This is the embedding vector for the query relation. For the first Embedding vectors of each label; The projection matrix is ​​learnable; This is the dimension scaling factor; The normalized exponential function is used to map the calculated similarity score to... arrive The attention weights between them are determined, and the sum of the weights of all labels is guaranteed to be equal to the sum of the weights of all labels. ; For the first The semantic importance score of each label relative to the current relationship.

5. The relation-guided fully inductive multimodal knowledge graph reasoning method according to claim 1, characterized in that, Encoding the local structural context to obtain local structural features includes: a set of local relation sequences. Each relation in the model is mapped to an embedding vector, which is then transformed using a multilayer perceptron and aggregated using average pooling to obtain local structural feature vectors. : in, This represents the pooling operation, used to aggregate multiple vectors into a single vector of fixed length. This represents a multilayer perceptron, used to perform feature transformation and nonlinear mapping on the relational features of the input; Represents a set of local relation sequences The embedding vector of any relation in the dataset; This refers to the set of local relation sequences of the target entity extracted in the aforementioned steps.

6. The relation-guided fully inductive multimodal knowledge graph reasoning method according to claim 1, characterized in that, The process of generating the initial representation of the entity includes: introducing an adaptive gating mechanism to dynamically adjust the contribution ratio of semantic features and structural features to generate an initial node representation. in, This represents a vector concatenation operation. It is the Sigmoid activation function. Here is the weight matrix of the gated network. These are the bias parameters for the gating system. For adaptive gating coefficients.

7. The relation-guided fully inductive multimodal knowledge graph reasoning method according to claim 1, characterized in that, The process of performing global message passing on the message passing network to obtain the enhanced final entity representation includes: using the generated fusion features As the initial features of the nodes, graph neural networks are used for... Layer feature updates yield the final entity representation. : in, Represents the target entity In the The updated feature representation of the layer network; This represents the node feature update function of a graph neural network; Represents the target entity In the Layer feature representation; Represents the neighborhood feature aggregation function; Representing neighboring nodes In the Layer feature representation; Represents the target entity The set of one-hop neighbor nodes in a message-passing network; Represents the target entity The The initial feature representation of the layer is the fusion feature generated in step S5.

8. The relation-guided fully inductive multimodal knowledge graph reasoning method according to claim 1, characterized in that, The calculation of the probability of the target triplet being true includes: using RotatE as the scoring function, for a given query... Calculate candidate tail entities Score: in, Indicates the entity by head ,relation and candidate tail entities The score for the formation of the triple; This represents the embedding vector corresponding to a given head entity; This represents the embedding vector corresponding to the query relationship; This represents the embedding vector corresponding to the candidate tail entity; Represents head entity The final entity representation vector obtained after the aforementioned global message passing; Represents candidate tail entities The final entity representation vector obtained after the aforementioned global message passing; It represents the Hadamardi (or Hadama) stack; Norms are used to measure the distance between vectors.

9. A relation-guided, fully inductive, multimodal knowledge graph reasoning system, characterized in that, The system includes: The semantic tag generation module acquires image data and text description data corresponding to the target entity to be processed in the knowledge graph, and generates structured semantic tags; the knowledge graph is constructed by entity recognition and relation extraction of knowledge, the knowledge is related knowledge in question answering system and semantic search, and the entity is text data obtained from natural language text using named entity recognition tools; The structural context extraction module extracts the local structural context of the target entity to be processed in the knowledge graph reasoning stage within the current reasoning graph. The relation-guided semantic feature construction module, guided by query relations, performs attention filtering on the structured semantic tags to construct a relation-guided semantic feature vector; The local structural feature encoding module encodes the local structural context to obtain local structural features; The gating fusion module, through an adaptive gating mechanism, fuses the relation-guided semantic features and the local structural features to generate an initial representation of the target entity; The entity representation enhancement module uses the initial representation as node features and performs global message passing on the message passing network to obtain the enhanced final entity representation. The reasoning scoring module calculates the probability of the target triplet being true based on the final entity representation and the query relationship representation, thus completing the reasoning.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 8.

Images